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FOREWORD 


The  U.S.  Army  has  made  a  substantia!  commitment  to  the  use  of  simulation  for  training, 
readiness,  concept  development,  and  test  and  evaluation.  Virtual  Environment  (VE)  technology, 
which  uses  position  tracking  and  real-time  update  of  visual,  auditory,  and  other  displays  (e.g., 
tactile)  has  the  potential  to  provide  simulation-based  training  for  dismounted  soldiers.  Since  VE 
technologies  are  relatively  new,  there  is  little  information  to  indicate  where  and  how  they  are 
best  used  for  dismounted  soldier  training.  The  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  has  established  a  research  program  to  determine  the  characteristics  of 
the  VE  technologies  needed  to  allow  dismounted  soldiers  to  fight  on  virtual  battlefields.  In 
addition  to  research  needed  to  determine  appropriate  VE  user  interfaces,  research  is  needed  to 
determine  appropriate  instructional  strategies  for  VE  training  applications.  This  research  should 
identify  ways  of  taking  advantage  of  the  unique  advantages  of  VEs  for  training,  and  for  working 
around  the  temporary  shortfalls  in  VE  technologies. 

This  report  describes  research  conducted  to  support  the  development  of  VE  systems  for 
dismounted  soldier  training.  The  results  of  the  research  provide  information  about  the 
characteristics  needed  for  effective  immersion,  learning,  and  performance  in  VEs.  The  results  of 
this  research  were  briefed  to  the  Office  of  the  Secretary  of  the  Army  for  Acquisition,  Research, 
and  Technology  (March,  2000),  Headquarters,  U.S.  Army  Training  and  Doctrine  Command 
(July,  1999),  and  the  Dismounted  BattleSpace  BattleLab  (November,  1999).  Information  about  a 
portion  of  this  research  has  also  been  included  in  the  proceedings  of  the  1998  IMAGE  Society 
Conference,  the  1998  and  1999  fiiterservice/Industry  Training,  Simulation,  and  Education 
Systems  Conference,  and  the  2000  Human  Factors  and  Ergonomics  Society  Conference.  An 
article  based  on  this  report  is  in  press  in  the  journal  Presence. 

The  ARI  Simulator  Systems  Research  Unit  conducts  research  to  improve  the 
effectiveness  of  training  simulators  and  simulations.  The  work  described  is  a  part  of  ARI  Work 
Package  202a,  VERITAS  (Virtual  Environment  Research  for  Infantry  Training  and  Simulation). 
VERITAS  is  the  ARI  portion  of  a  Science  and  Technology  Objective,  Virtual  Environments  for 
Dismounted  Soldier  Simulation,  Training  and  Mission  Rehearsal,  which  is  being  carried  out  in 
conjunction  with  the  U.S.  Army  Simulation,  Training,  and  Instrumentation  Command  and  the 
U.S.  Army  Research  Laboratory. 


^TA  M.  SIMUTIS 
Technical  Director 
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INSTRUCTIONAL  STRATEGIES  FOR  TRAINING  TEAMS  IN  VIRTUAL 
ENVIRONMENTS 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  U.S.  Army  has  made  a  substantial  commitment  to  the  use  of  virtual  environment 
(VE)  technology  to  create  virtual  battlefields  for  combat  training  and  mission  rehearsal, 
development  of  military  doctrine,  and  evaluation  of  weapon  system  concepts  prior  to  acquisition 
decisions.  All  of  these  functions  would  be  improved  by  a  better  representation  of  dismounted 
soldiers  on  the  virtual  battlefield.  The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ART)  has  established  a  research  program  to  determine  the  characteristics  of  the 
VE  technologies  needed  to  allow  dismounted  soldiers  to  fight  on  virtual  battlefields.  In  addition 
to  research  needed  to  support  appropriate  VE  user  interfaces,  research  is  needed  to  determine 
appropriate  instructional  strategies  for  tapping  the  unique  advantages  of  VEs  for  training,  and  for 
mitigating  the  temporary  shortfalls  in  VE  technologies. 

Procedure: 

Functional  requirements  were  generated  for  mission  tasks  and  environments  that  would 
support  VE  team  training  research.  The  missions  combined  task  elements  of  urban  search  and 
rescue.  Special  Weapons  and  Tactics  Teams  (SWAT),  and  infantry  operations  inside  buildings. 
These  requirements  were  used  to  design  and  implement  the  Fully  Immersive  Team  Training 
(FITT)  research  system.  The  FITT  was  used  in  an  experiment  to  examine  instructional  strategies 
for  team  training  in  VEs.  In  that  experiment,  94  participants  recruited  from  local  colleges  served 
as  two-person  teams  in  conducting  search  missions.  After  reading  the  mission  training  manual, 
teams  received  guidance  on  performing  the  missions  either  before  (Demonstration),  during 
(Coaching),  or  after  (after-action  Replay  with  critique)  practicing  the  mission  tasks,  or  not  at  all 
(Control  group).  After  practice,  each  team  completed  a  test  mission  without  guidance. 

Findings: 

Results  indicated  that  the  FITT  adequately  creates  immersive  environments  such  that  participants 
can  function  as  teams  in  learning  and  performing  search,  communication,  and  security 
procedures  for  training  research  purposes.  In  particular,  the  FITT  interface  for  walking  through 
the  simulated  environments  was  learned  quickly  by  the  participants.  Each  of  the  instructional 
strategies  could  be  implemented  with  the  FITT.  Both  the  Control  and  the  Instructional  Strategy 
groups  (Demonstration,  Coaching,  and  Replay)  improved  with  practice  on  some  performance 
measures.  For  the  test  mission  performance  measures,  differences  between  the  Control  group  and 
the  Instructional  Strategy  groups  were  not  statistically  significant,  nor  were  the  differences 
among  the  Instructional  Strategy  groups.  Despite  the  brevity  of  the  VE  immersions,  about  eight 
minutes  per  mission  followed  by  a  rest  break,  simulator  sickness  was  a  problem  in  that  about 
nine  percent  of  the  participants  withdrew  before  completion  of  the  experiment. 


Utilization  of  Findings: 

Technical  lessons  learned  from  this  experiment  were  used  in  the  development  of  a 
follow-on  system  that  will  be  used  in  research  to  examine  distributed  team  training  in  which  the 
participants  are  at  remote  sites.  Lessons  learned  concerning  the  instructional  strategies  will  be 
incorporated  in  a  follow-on  effort  to  develop  tactical  training  for  small  unit  infantry  leaders. 

The  FITT  interface  for  locomotion  (walking)  through  VE  should  be  of  value  in  many  VE 
applications,  including  training,  architectural  design,  education,  and  recreation. 
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INSTRUCTIONAL  STRATEGIES  FOR  TRAINING  TEAMS 
IN  VIRTUAL  ENVIRONMENTS 

This  report  describes  the  design  and  implementation  of  the  Fully  Immersive  Team 
Training  (FITT)  research  system,  and  the  first  experiment  conducted  with  the  system.  FTTT  was 
developed  to  support  research  on  the  use  of  distributed  Virtual  Environments  (VEs)  for  training 
dismounted  infantry.  The  research  is  focused  on  VE  training  applications  for  small  units  and  unit 
leaders. 


The  report  is  organized  as  follows;  the  background  section  briefly  describes  the  military 
training  requirement  that  this  research  addresses,  and  outlines  our  previous  research  in  this  area. 
Next,  an  in-depth  description  of  the  design  and  implementation  of  the  FITT  research  system  is 
presented.  The  final  major  section  of  the  report  describes  the  design,  procedure,  results  and 
discussion  of  the  first  experiment  conducted  with  the  FITT,  and  implications  for  future  research. 

Background 

The  U.S.  Army  has  made  a  substantial  investment  in  the  use  of  VEs  for  training.  The 
SIMNET  program  (Alluisi,  1991)  and  initial  efforts  with  the  Close  Combat  Tactical  Trainer 
focused  primarily  on  collective  training  involving  combat  vehicles  such  as  tanks  and  infantry 
fighting  vehicles.  There  is  now  a  growing  effort  to  develop  methods  to  use  immersive  VEs  to 
train  soldiers  who  fight  on  foot,  such  as  infantry  and  special  operations  forces.  The  expanding 
interest  in  inserting  the  individual  combatant  into  the  virtual  battlefield  is  driven  by  a  number 
of  factors,  including  the  recognition  that  members  of  small  dismounted  units  will  fece  greater 
responsibilities  and  challenges  in  both  combined  arms  combat  and  in  contingency  operations. 

In  addition  to  supporting  Army  training  requirements,  individual  virtual  simulation  could 
support  mission  planning  and  rehearsal.  The  capability  to  insert  the  individual  combatant  into 
a  virtual  combined  arms  battlefield  could  also  be  used  in  the  conceptualization,  design,  and 
testing  of  new  equipment,  doctrine,  and  organization. 

VE  Training  Considerations 

The  National  Research  Council’s  Conrniittee  on  Virtual  Reality  Research  and 
Development  has  identified  numerous  ways  in  which  VE  has  tremendous  potential  as  a  training 
medium  (Durlach  &  Mavor,  1995).  For  example,  it  can  be  expected  that  VE-based  training  will 
eventually  offer  the  benefits  of  traditional  simulator-based  training  such  as  safety,  flexibility, 
repeatability,  and  cost  effectiveness.  In  addition,  for  some  situations  VE  training  may  prove  more 
effective  than  conventional  training  in  the  physical  world.  VEs  offer  rich  learning  possibilities 
such  as  providing  multiple  viewpoints  and  various  levels  of  abstraction  that  cannot  be  created 
with  conventional  training.  However,  the  Council  pointed  out  that  current  (as  of  1995)  VE  work 
in  training  was  at  the  research  and  development  stage.  Challenges  to  the  use  of  VE  for  training 
include  not  only  inadequacies  of  the  currently  available  VE  technologies  but  also  a  lack  of 
knowledge  concerning  basic  psychological  issues  related  to  training  and  training  transfer.  Thus, 
maximizing  positive  transfer  from  VE  training  to  real-world  performance,  and  avoiding  negative 
transfer,  requires  not  only  improvements  in  VE  technology  but  also  in  our  knowledge  of  how  to 


1 


design  and  use  VE  training  systems.  Johnson,  Rickel,  Stiles,  and  Munro  (1998)  also  present 
compelling  arguments  that  the  development  of  effective  VE  training  systems  must  involve  more 
than  considerations  of  the  fidelity  of  the  renderings  and  accuracy  of  simulated  behaviors. 

ARI/IST  VE  Training  Research 

Since  1992,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI),  in  conjunction  with  the  University  of  Central  Florida’s  Institute  for  Simulation  and 
Training  (1ST),  has  undertaken  a  program  of  behavioral  science  research  to  investigate  the  use  of 
VE  technologies  to  support  training  of  dismounted  soldier  tasks.  The  research  has  been 
concerned  primarily  with  three  issues:  (1)  how  the  characteristics  of  the  individual’s  interface 
with  the  VE  affect  task  performance  and  the  acquisition  of  new  skills,  (2)  how  skills  can  best  be 
trained  in  VEs  and  how  well  they  transfer  to  the  real  world,  and  (3)  the  incidence  and  severity  of 
the  side  effects  that  can  result  from  immersion  in  VEs.  To  date  research  in  this  program  has 
examined  VE  display  and  input  device  requirements  for  training  tasks  such  as  visual  tracking, 
object  manipulation,  locomotion,  distance  estimation,  route  learning  in  buildings,  building 
search,  and  land  navigation.  Variables  investigated  included  the  t)q)e  of  control  device,  amount 
of  practice  on  the  tasks,  stereoscopic  vs.  monoscopic  head-mounted  displays  (HMDs),  and  type 
of  display  device  (monitor.  Binocular  Omni-Orientation  Monitor,  or  HMD).  Under  this  program, 
thirteen  experiments  involving  over  500  human  participants  have  been  conducted.  This  research 
is  summarized  in  Knerr  et  al.  (1998). 

Development  of  the  FITT  Functional  Requirements 

Previous  ARI  VE  training  research  involved  immersing  individuals  one  at  a  time. 
Developing  a  system  for  teams,  the  FITT  system,  presented  a  number  of  challenges.  We 
required  a  system  in  which  two  individuals  (with  potential  for  expansion  to  larger  teams) 
could  be  immersed  simultaneously  and  engage  in  meaningful  team  activities.  In  addition,  we 
required  a  system  that  could  support  instructional  features  and  interventions  beyond  the 
simulation  of  real-world  conditions. 

For  our  initial  research  in  using  VEs  to  train  teams,  we  wanted  tasks  that  would  require 
rapid  decision  making  in  complex,  dynamic  environments;  situation  awareness;  and  teamwork. 

A  challenge  was  to  develop  tasks  that  could  be  implemented  with  current  VE  technology  and 
represented  the  critical  elements  of  military  tasks,  yet  were  not  beyond  the  grasp  of  participants 
with  no  previous  military  training.  (For  initial  experiments  with  new  systems,  we  recruit  our 
research  participants  from  local  colleges.). 

To  develop  the  tasks  we  reviewed  field  manuals  for  infantry  combat  in  built-up  areas, 
urban  search  and  rescue  procedures,  and  training  videotapes  for  Special  Weapons  and  Tactics 
Team  procedures.  From  these,  we  developed  procedures  that  are  generalized  versions  of  tasks 
and  activities  that  would  be  performed  during  emergencies  such  as  urban  search  and  rescue 
missions,  hostage  rescue,  or  disasters  involving  hazardous  materials  (HAZMAT). 

The  tasks  are  organized  as  a  mission  in  which  a  two-person  team  searches  the  rooms  of  a 
building  for  canisters  containing  hazardous  gas.  The  team  carries  equipment  to  determine  if  a 


canister  contains  gas  and  to  deactivate  the  canisters  if  necessary.  Each  team  member  is 
represented  in  the  VE  by  an  avatar.  The  avatars  wear  protective  clothing  and  a  breathing 
apparatus,  and  carry  a  tranquilizer  dart  gun  that  can  temporarily  incapacitate  the  enemy.  The 
search  is  complicated  by  the  threat  of  interference  by  enemy  forces. 

Team  member  #1  is  the  designated  Team  Leader.  The  Team  Leader  directs  team 
movement  to  provide  an  efficient  search  while  maintaining  team  security.  The  leader  radios 
reports  of  canister  deactivations  and  encounters  with  the  enemy  to  an  off-site  mission 
commander.  In  addition  to  a  tranquilizer  dart  gun,  the  Team  Leader  carries  a  paint  marker  used 
to  mark  the  doorways  of  rooms  that  have  been  searched.  Team  member  #2,  the  HAZMAT 
Equipment  Specialist,  carries  a  dart  gun  and  a  device  to  detect  and  deactivate  active  canisters. 
The  tools  reflect  the  roles  of  the  team  members:  a  successful  mission  requires  that  team  members 
cooperate  with  each  other. 

The  missions  are  situated  in  a  ten-room  building.  Computer-generated  enemy  and 
innocent  bystanders  move  through  hallways  and  rooms.  The  presence  of  both  enemy  and  neutral 
forces  require  rapid  shoot/don’t  shoot  decisions.  Enemies  can  be  either  lightly  armed  looters  or 
heavily  armed  terrorists,  requiring  the  trainee  teams  to  prioritize  targets. 

Mission  instructions  specify  the  amount  of  air  available  for  each  mission.  Team  members 
must  remember  to  periodically  check  their  remaining  air  supply  indicators,  and  must  decide 
when  to  begin  exiting  the  building.  This  is  not  an  easy  decision  in  that  leaving  too  soon  wastes 
search  time,  but  underestimating  the  time  needed  to  exit  the  building  results  in  mission  failure. 

Procedures  include  rules  for:  the  order  in  which  rooms  are  searched,  team  formation  for 
room  entry,  actions  on  contact  with  enemy  and  innocent  bystanders,  assigned  areas  of 
responsibility  within  a  room,  and  how  and  what  to  report  on  the  radio  network.  Successful 
performance  of  some  of  the  procedures  requires  the  team  members  to  coordinate  their 
movements  and  actions  to  within  a  second  of  each  other. 

A  training  manual  was  developed  to  introduce  naive  participants  to  the  mission 
procedures.  The  2,300  word,  thirteen-page  training  manual  includes  a  mission  overview,  learning 
objectives,  task  descriptions  with  graphics,  and  ends  with  a  nmemonic  to  help  the 
participant/trainee  remember  the  procedures.  The  manual  does  not  assume  that  the  reader  has  any 
previous  training  or  experience  directly  relevant  to  the  mission  tasks.  The  Flesch-Kincaid 
readability  score  of  7.4  for  the  manual  falls  within  the  range  recommended  for  manuals  for  the 
armed  services  (MIL-M-38784B,  1983).  The  training  manual  for  the  Team  Leader  is  at 
Appendix  B.  (The  actual  manuals  presented  to  the  participants  were  printed  in  color.)  A  paper 
and  pencil  knowledge  test  was  developed  to  measure  each  participant’s  mastery  of  the  manual. 

The  two-person  FITT  team  meets  the  defining  characteristics  of  teams  listed  by  Salas, 
Dickinson,  Converse,  and  Tannenbaum  (1992)  in  their  review  of  team  training  research:  there 
is  some  organizational  structure  of  the  team  members;  each  individual  team  member  has 
specific  tasks  or  functions;  and  there  is  a  common  goal,  mission,  or  objective. 


WIT  Design 


FITT  incorporates  a  number  of  modular  components  to  provide  a  flexible  framework  for 
developing  networked  team  training  applications.  The  primary  components  of  this  system  are  the 
individual  combatant  simulator  (ICS),  a  computer-generated  entity  server  (CGES),  a  mission 
controller/experimenter  station,  an  audio  system  to  simulate  radio  communication  between  the 
participants,  and  a  system  for  data  collection  and  mission  playback. 

Hardware  Configuration 

FITT  consists  of  the  following  hardware  components: 


ICS  #1: 

•  8  Processor  /  3  Pipe  Silicon  Graphics  Onyx™  RE2  (Team  Leader  and  CGES) 

•  6  Tracker  Ascension  MotionStar™ 

•  Virtual  Research  VR4™  Head  Mounted  Display 

ICS  #2: 

•  4  Processor  /  1  Pipe  Silicon  Graphics  Onyx™  RE2  (Specialist) 

•  6  Tracker  Ascension  Flock  of  Birds™ 

•  Virtual  Research  VR4  Head™  Mounted  Display 

Silicon  Graphics  Indigo2™  High  Impact  (Stealth) 

Silicon  Graphics  Indy™  (Data  Collection) 

Dell  Pentium  90  (Audio  Capture) 

Dell  486  (Startup  Browser) 

Misc.  handheld  input  devices 

Misc.  video  equipment  for  observing  player  views  and  videotaping  after  action  critiques. 

This  hardware  list  is  used  for  the  current  two  player  configuration  of  FITT.  The  configuration  is 
easily  scalable,  and  extra  ICSs  could  be  added,  as  necessary,  to  support  more  players  in  the  VE. 
Note  that  the  extra  processing  power  of  the  8  processor  Onyx  allows  it  to  run  the  CGES 
component,  in  addition  to  its  duties  as  an  ICS. 

Individual  Combatant  Simulator 

The  ICS  provides  a  link  between  the  participant  and  the  networked  VE.  This  simulator 
provides  a  view  into  the  shared  virtual  world  via  an  image  generator  and  a  viewpoint-tracked 
display  device.  The  ICS  also  provides  a  means  for  interacting  directly  with  the  virtual  world 
through  a  variety  of  Input/Output  devices  that  can  be  selected  according  to  their  applicability  to 
the  mission  scenario.  Finally,  it  is  the  role  of  the  ICS  to  translate  the  real-world  gestures  and 
movements  of  the  participant  into  appropriate  avatar  responses. 
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Sensor  Configuration 


Each  team  member  is  suited  with  six  position  sensors  (Figure  1)  used  to  determine  body 
position,  view,  and  locomotion.  As  can  been  seen  in  Figure  2,  the  FITT  system  can 
accommodate  a  wide  range  in  participant  height  and  morphology.  The  sensor  mounted  on  the 
HMD  determines  gaze  direction.  Body  direction  is  measured  by  the  position  of  a  sensor  mounted 
on  a  lightweight  wooden  backpack  worn  by  each  participant.  This  sensor  determines  which 
direction  the  participant’s  torso  is  facing,  and  which  direction  he  or  she  will  walk  when  moving 
forward.  The  back  pack  also  acts  as  a  cable  guide  for  the  wires  that  run  to  the  sensors. 


Figure  1.  Sensor  configuration. 

Lx)comotion  method.  The  ankle  trackers  are  used  for  locomotion  through  the  VE.  Stride 
length  is  determined  by  the  amount  of  time  the  angle  sensor  is  held  above  a  defined  height. 
Slowly  marching  in  place,  raising  and  lowering  the  feet,  provides  a  smooth  gedt  through  the  VE. 
The  software  object  responsible  for  ambulating  the  participant  watches  the  step  height  by 
comparing  the  height  of  the  ankle  trackers  relative  to  the  base  of  the  ICS.  When  the  tracker 
crosses  a  software-defined  threshold,  a  step  is  initiated  and  the  height  of  the  step  is  translated 
into  pitch  rotations  for  the  hip  and  knee  joints,  allowing  the  avatar  to  raise  and  lower  its  legs  as 
the  player  walks.  The  locomotion  model  also  allows  discrete  backward  steps. 

Arm  articulation.  The  right  arm  of  each  player  is  tracked  in  order  to  articulate  the  arm  of 
the  virtual  avatar.  This  requires  two  sensors  working  in  conjunction  with  the  back  sensor.  The 
position  of  the  player’s  shoulder  joint  is  determined  by  offset  from  the  back  tracker.  This 
phantom  shoulder  position  is  used  with  a  sensor  strapped  to  the  elbow  to  derive  a  vector  for  the 
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upper  arm.  An  additional  sensor  mounted  on  the  input  device  and  held  within  the  hand  of  the 
player  provides  a  way  of  deriving  the  vector  for  the  lower  arm  position. 


Figure  2.  Individual  combatant  simulators. 

Despite  the  existence  of  methods  for  animating  avatar  arm  movement  using  only  a  back 
sensor  and  a  hand  (Tolani,  Deepak,  &  Badler,  1996),  the  FITT  development  team  chose  the 
expense  of  an  extra  sensor  for  the  finer  model  granularity  offered.  In  addition,  a  fully  rotational 
model  of  the  upper  and  lower  arm  provides  a  mechanism  to  implement  complex  hand  and  arm 
signals. 


Manual  control  device.  The  initial  objective  of  the  design  team  was  to  provide  a  manual 
controller  device  that  approximated  the  mass  and  size  or  “heft”  of  an  actual  pistol  or  comparable 
item  of  equipment.  Informal  pilot  testing  quickly  revealed  that  after  only  a  few  minutes  into  a 
mission,  the  weight  of  the  controller  became  distracting.  A  very  light  controller  was  eventually 
settled  upon:  the  palm  grip  of  a  Gravis  Blackhawk  joystick  (Figures  3  &  4).  The  controller 
thumb  switch  advances/cycles  through  an  array  of  configurable  hand  held  items,  beginning  with 
an  empty  hand,  then  the  sidearm,  and  then  through  the  tools  available  to  the  team  member. 

(Team  Leaders  were  equipped  with  paint  markers,  and  Equipment  Specialists  were  outfitted  with 
gas  canister  detector/deactivators.)  The  index  finger  is  held  over  the  ‘trigger’. 
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Figures.  Hand  controller  input  device. 


Figure  4.  Controller  in  use. 


Computer-generated  Entity  Server 

The  CGES  provides  support  for  all  of  the  simulated  entities  in  the  VE.  Environmental 
attributes  that  can  be  expressed  as  a  Boolean  state  variable  (such  as  doors  left  open  or  closed,  or 
lights  turned  on  or  off)  can  be  rapidly  configured  and  implemented  in  FITT.  For  example,  gas 
canisters  encountered  in  the  VE  can  be  found  in  one  of  two  states:  active  or  “dud”.  In  addition  to 
serving  simple  two-state  environmental  elements,  CGES  can  also  be  used  to  serve  more 
sophisticated  entities  such  as  computer-generated  enemies,  referred  to  as  Opposing  Forces 
(OPFOR).  Although  configuration  of  these  types  of  dynamic  elements  is  not  as  quickly 
implemented  as  Boolean  elements,  (they  generally  require  the  development  of  custom  behavior 
code)  allowances  have  been  made  for  a  broad  range  of  expression  on  the  part  of  the  mission 
designer.  The  OPFOR  can  be  programmed  with  various  levels  of  hostility  and  ability. 
Configurable  parameters  for  OPFOR  include  span  of  peripheral  vision,  response  time  for 
orienting  toward  and  firing  at  a  participant  after  sighting  a  mission  participant,  sentry  path,  and 
accuracy  with  a  weapon.  For  example,  OPFOR  can  vary  from  an  armed  terrorist  with  pinpoint 
firing  accuracy  and  a  high  degree  of  visual  acuity,  to  a  lightly  armed  looter,  to  an  unarmed 
innocent  bystander  who  should  not  be  fired  at  when  sighted.  The  OPFOR  can  also  be  set  to  stand 
in  place  and  guard  an  area,  or  can  be  given  a  sentry  path  to  follow  and  search  for  intruders  (that 
is,  the  participants/trainees).  Each  OPFOR  reacts  to  being  shot  by  the  players’  tranquilizer  pistols 
by  slumping  to  the  ground,  and  not  moving  for  the  remainder  of  the  simulation. 


Mission  Commander/Experimenter  Station 

The  FITT  system  mission  conunanderXexperimenter  station  was  developed  with  features 
to  allow  a  single  individual  to  serve  as  the  mission  commander  and  as  an  experimenter.  As 
commander,  appropriate  communications  are  exchanged  with  the  participant  team  members  over 
the  simulated  radio  network.  The  station  also  supports  experimenter  tasks  such  as  scoring  team 
performance  and  instructional  interventions  such  as  coacMng.  A  single  keystroke  toggles  the 
display  between  a  top-down  view  of  the  entire  building  that  the  team  is  searching  and  a  zoomed- 
in  close-up  view  centered  over  the  team  leader.  To  allow  the  commander  to  discern  more  easily 
which  equipment  is  in  use  at  a  given  time,  the  equipment  in  use  by  each  team  member  is  visually 
represented  much  larger  than  scale  relative  to  the  HAZMAT-suited  avatars.  Along  with  the 
graphical  display  of  the  players’  positions  in  the  VE,  statistics  are  displayed  which  quantify  the 
actions  of  the  participants  and  assist  the  experimenter  in  determining  the  efficiency  of  the  team 
as  they  move  through  their  required  tasks.  For  example,  the  display  includes  the  number  of  times 
each  participant:  has  been  engaged  by  enemy  fire  during  a  mission,  has  changed  equipment,  and 
has  collided  with  something  in  the  VE. 

Audio  system 

Audio  conununication  is  an  essential  ingredient  of  FITT.  The  audio  for  each  player 
integrates  environmental  sounds  such  as  gun  shots,  door  openings  and  closings,  and  coUision 
sounds,  as  well  as  communication  with  the  other  two  members  of  the  team.  Because  the  ICSs  are 
in  separate  rooms,  the  participants  can  speak  to  each  other  during  the  mission  only  over  a 
simulated  radio  net.  This  avoids  the  problem  that  if  the  ICSs  were  in  the  same  room  the  direction 
of  voice  would  not  change  appropriately  as  the  participants’  avatars  changed  their  relative 
positions  to  each  other.  Microphones  attached  to  the  HMDs  at  the  Leader  and  Specialist 
positions  feed  their  signals  to  the  master  mixing  console,  where  the  Commander’s  microphone 
signal  is  included  in  the  mix.  The  Commander  can  talk  with  each  of  the  other  team  members 
separately  if  the  need  arises.  The  signal  from  the  master  mixer  is  then  sent  back  to  each  player 
where  a  small  mixer  integrates  it  with  enviromnental  sounds  generated  from  audio  files  on  the 
ICS  machines.  These  final  signal  mixes  are  then  delivered  to  the  headsets  of  the  HMDs. 

Audio  Capture  and  playback  is  performed  by  a  separate  machine,  a  Dell  Optiplex  560  PC 
running  Windows  95.  The  PC’s  sound  hardware  (a  Sound  Blaster™  AWE64)  is  connected  to  an 
intercom  system  that  allows  both  players  and  the  experimenter  to  talk  to  each  other  as  if 
communicating  via  radio.  The  PC  is  also  attached  via  serial  cable  to  both  the  SGI  Onyx  running 
the  playback  system,  and  the  SGI  Indy  running  the  data  capture  program.  During  data  capture, 
the  PC  receives  its  recording  commands  from  the  Indy.  The  Indy  first  instmcts  the  PC  to  ready 
itself  for  audio  capture  and  informs  it  of  the  current  trial’s  unique  ID  number.  Next,  it  instructs 
the  PC  to  begin  audio  capture.  At  this  point  the  PC  begins  “listening”  on  its  audio  input  port  for 
audio  communication.  When  the  audio  level  rises  above  a  preset  threshold,  the  PC  saves  the 
current  time  (from  its  internal  multimedia  timer)  and  begins  recording  audio  into  a  .wav  file.  The 
threshold  code  acts  as  a  software  voice  activated  circuit,  and  reduces  the  size  of  the  .wav  files 
when  no  communication  is  occurring.  After  the  audio  level  falls  below  the  threshold  for  a  fixed 
amount  of  time,  recording  stops.  The  PC  calculates  the  length  of  the  audio  clip  and  saves  the 
ending  time  of  the  clip.  It  then  returns  to  “listening”  mode.  Finally,  the  SGI  Indy  informs  the  PC 
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when  to  stop  recording  audio.  All  audio  clips  with  their  associated  time  stamps  are  then  saved 
into  a  file  in  the  same  directory  as  the  .wav  files.  This  allows  them  to  be  recalled  later  by  the 
playback  system. 

Data  Capture  and  Playback  System 

During  any  mission,  the  data  collection  system  records  all  of  the  network  traffic.  This 
recording  can  be  used  to  replay  events  as  they  unfolded  during  a  particular  mission,  or  it  may  be 
analyzed  to  produce  a  numerical  summary  for  performance  evaluation.  Along  with  recording  the 
network  traffic,  the  data  collection  system  also  records  all  of  the  voice  communications  among 
the  participants  in  the  exercise.  When  the  data  stream  is  played  back,  the  audio  of  those 
communications  is  also  played  to  provide  synchronized  audio  and  visual  replay  of  the  mission. 

Data  recording  is  accomplished  by  capturing  the  Protocol  Data  Unit  (PDU)  traffic  from 
all  participating  machines  (both  player  machines  and  the  CGES  server)  into  a  terse  binary  file. 
The  data  capture  program  runs  on  an  SGI  Indy  workstation  attached  to  the  network.  The  start  of 
data  recording  is  signaled  by  a  Start/Resume  PDU,  and  the  end  is  signaled  by  a  Stop/Freeze 
PDU.  These  are  sent  by  the  experimenter  via  the  Stealth  machine’s  keyboard,  to  all  machines  at 
the  beginning  of  the  experimental  trial.  AH  PDUs  are  time  stamped  by  the  sending  machine  and 
all  machines  are  time  slaved  to  one  designated  master  time  server  during  each  experimental  run. 
This  assures  that  all  data  can  be  reconstructed  in  the  proper  sequence  during  playback. 

The  Playback  system  is  designed  to  allow  experimenters  and  participants  to  view  a 
complete  audio/visual  computer-generated  reenactment  of  an  experimental  trial  (mission). 
Features  of  this  system  include: 

•  The  ability  to  view  any  instant  of  the  experiment  from  any  viewpoint 

•  The  ability  to  advance  or  rewind  the  reenactment  at  any  desired  rate  of  speed 

•  The  accurate  representation  of  environment  changes  (such  as  doors  opening  and  paint  marks 
on  walls)  both  during  normal  playback  and  fast/slow  motion 

•  The  recreation  of  audio  cues  ( door  sounds  for  example)  and  player  audio  communications 
synchronized  to  the  visual  playback 

The  Playback  system  runs  on  either  of  the  SGI  Onyx  RealityEngine2  machines  used  as 
ICS  stations  during  the  mission.  The  software  is  built  from  the  same  libraries  used  to  develop  the 
experiment.  In  effect,  it  acts  as  a  stealth  station  with  a  variable  viewpoint,  but  reads  PDUs  out  of 
a  data  capture  file  rather  than  fresh  off  the  network.  In  addition,  the  same  PC  used  to  record  the 
participants’  audio  communication  is  used  to  play  back  this  audio,  allowing  the  experimenters 
and  participants  to  both  hear,  as  well  as  see,  how  the  mission  progressed.  The  control  panel  for 
the  viewer  has  familiar  VCR-style  controls  allowing  play,  stop,  rewind,  and  fast-forward  at  the 
click  of  a  mouse.  There  is  also  a  shuttle  dial  that  allows  for  variable  speed  playback,  both  slower 
and  faster  than  normal.  The  panel  provides  a  real-time  clock  and  a  PDU  counter  to  indicate 
where  in  the  recorded  mission  the  playback  system  is  ciurently  pointing. 


9 


The  playback  system  begins  by  first  reading  in  the  requested  PDU  file,  sorting  the  PDUs 
chronologically  (by  their  saved  time  stamp),  and  pre-loading  the  appropriate  body  and 
environment  models.  It  then  establishes  communications  with  the  audio  playback  PC,  instructing 
it  to  prepare  to  play  the  appropriate  trial’s  audio  data.  When  the  play  button  is  pressed  on  the 
Playback  control  panel,  the  PC  is  sent  a  “play”  command  with  the  current  time  index  (in 
milliseconds).  The  PC  then  computes  which  .wav  file  to  play  based  on  this  time  index.  If  the 
time  happens  to  fall  in  between  two  .wav  files  in  its  list,  it  waits  until  the  correct  time  to  play.  If 
the  time  is  somewhere  within  a  .wav  file,  it  begins  playback  from  an  appropriate  offset  into  the 
file.  Audio  is  only  played  while  the  system  is  in  retd-time  playback  mode;  therefore,  if  any 
control  other  than  “play”  is  selected  on  the  control  panel,  the  audio  PC  is  instructed  by  the  SGI  to 
stop  playback. 

The  simulation  time  is  computed  by  the  SGI  running  the  visual  portion  of  the  playback 
system.  If  the  playback  system  is  in  “play”  mode,  the  system’s  real-time  clock  is  used  to  update 
the  simulation  time  each  frame.  This  provides  an  accurate  representation  of  the  passage  of  time. 
If  it  is  in  a  fast-wind  or  slow-motion  mode,  the  time  is  arbitrarily  incremented  or  decremented. 
Each  frame,  the  SGI  reads  all  PDUs  from  the  file  that  happened  prior  to  (or  if  rewinding, 
subsequent  to)  the  current  simulation  time.  As  each  PDU  is  read,  the  appropriate  entities  are 
updated,  the  appropriate  weapons  fired,  and  the  appropriate  doors  opened  or  closed.  After  all 
necessary  PDUs  are  processed,  the  next  frame  is  drawn  and  the  cycle  repeated.  Originally,  a 
method  was  to  be  implemented  that  would  continuously  synchronize  the  graphics  data  to  the 
audio,  because  the  audio  capture  PC  is  not  time  slaved  to  the  primary  simulation  machines. 
Preliminary  testing  demonstrated  that  this  was  not  necessary,  however,  as  both  the  PC’s  and  the 
SGI’s  internal  clocks  were  independently  accurate  enough  to  control  the  timing  of  their 
respective  playbacks.  That  is,  no  discrepancy  in  the  audio  and  visual  data  was  perceived  over  the 
length  of  any  trial’s  playback. 

Avatars 


FITT  provides  a  means  for  representing  participants  within  the  shared  virtual  world  as 
avatars.  As  shown  in  Figure  5,  the  avatar  of  each  team  member  depicts  a  person  in  HAZMAT 
gear:  a  bulky  chemical  protective  suit  with  a  hehnet  containing  breathing  apparatus.  Thus,  the 
limitations  of  the  FTTT  to  represent  locomotion,  vision,  audition,  and  haptic  factors  enhance, 
rather  than  detract  from,  the  sense  of  presence.  For  example,  the  participants  are  told  that  the 
HAZMAT  suit  and  body  armor  provide  protection  but  limit  speed  of  movement  to  walking  and 
prevent  ruiming,  crawling,  or  jumping. 

Loftin  (1998)  described  a  team  training  VE  for  which  high  fidelity  human  models  were 
developed  to  serve  as  avatars  for  each  of  the  actual  participants  in  VE  training.  Because  our 
research  involves  hundreds  of  different  participants  it  is  not  practical  to  develop  a  model  for 
each  individual  participant.  Therefore,  an  additional  advantage  to  using  the  HAZMAT  suited 
avatars  is  that  they  do  not  depict  body  morphology,  gender,  or  ethnicity. 

The  3D-avatar  model  incorporates  45  degree  of  freedom  beads  allowing  for  realistic 
deflection  of  limbs  and  torso.  Setting  the  proper  rotation  angles  for  these  degree  of  freedom 
beads  is  the  responsibility  of  the  ICS. 
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Figure  5.  Avatars  in  HAZMAT  suits. 


Virtual  Environments 


The  basic  floor  plan  for  the  search  mission  consists  of  10  rooms  positioned  along  a 
hallway  containing  one  ninety  degree  turn.  Six  of  these  floor  plans  were  developed.  Each 
contains  the  same  square  footage  to  be  searched  but  they  differ  in  the  directions  in  which 
hallways  branch  and  the  locations  of  the  rooms.  Thus,  across  different  missions  the  search  task  is 
of  equivalent  difficulty  but  the  layout  of  the  building  can  not  be  memorized.  A  seventh  floor  plan 
has  also  been  created  which  involves  a  multistory  structure  and  presents  a  much  more 
challenging  search  task. 

Two  separate  VEs  were  designed  to  provide  practice  in  walking  in  a  VE  and  to  practice 
using  the  manual  control  device  to  select  and  use  equipment.  The  first  consists  of  a  large  room 
and  a  connecting  series  of  corridors.  The  large  room  contains  a  pattern  at  eye-height  on  which 
the  participant  can  focus  as  a  reference  while  adjusting  the  HMD.  The  corridors,  designed  after 
the  Virtual  Environment  Performance  Assessment  Battery  (VEPAB)  hallways  task  (Lampton,  et 
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al.  1994),  provide  practice  walking  under  more  demanding  conditions  and  allow  speed  and 
accuracy  (collisions)  measures  for  each  participant.  The  second  VE  contains  examples  of  the 
various  types  of  equipment  and  friendlies,  enemies,  and  neutrals  that  can  be  encountered  during 
the  missions.  In  this  VE  the  participants  separately  practice  using  the  pistol  and  the  equipment 
vmique  to  their  role  in  the  team. 


Networking  Issues 

The  FITT  is  a  networked  simulation  environment  that  uses  the  Distributed  Interactive 
Simulation  (DIS)  protocol  to  pass  messages  back  and  forth.  A  subset  of  DIS  protocol  version 
2.0.4  has  been  implemented  for  this  project.  Fully  articulated  human  figures  are  not  explicitly 
supported  by  the  DIS  protocol.  For  FITT,  body  and  limb  position  are  transmitted  via  articulated 
parts  structures  attached  to  the  Entity  State  PDU.  The  following  DIS  PDUs  are  used  during  the 
application: 

Entity  State  PDU  This  is  the  primary  PDU  used  for  the  player  avatars  and  the  computer¬ 
generated  opposition  forces.  The  articulated  parts  array  is  used  to  broadcast  the  position  and 
posture  of  the  avatar.  A  total  of  47  articulated  part  parameters  are  available  for  modification 
during  the  simulation. 

Fire  PDU  -  Detonate  PDU  These  two  PDUs  are  used  for  the  paint  marker  tool  and  the 
Gas  Cylinder  de-activator  tool,  as  well  as  for  the  tranquilizer  dart  pistols  available  to  team 
members. 

Action  Request  PDU  Door  opening  and  closing  events  are  broadcast  using  these  PDUs. 

Start  PDU  -  Stop  PDU  Used  by  the  menu  software  for  starting  and  stopping  the 
simulation. 

Collision  PDU  Used  by  the  Data  capture  hardware  to  log  collisions  for  playback. 

One  important  aspect  of  the  FITT  software  is  that  the  DIS  PDU  traffic  is  handled  by  an 
Entity  Services  application  that  runs  as  its  own  process  on  each  network  machine.  All  message 
queuing  and  dead  reckoning  for  all  PDUs  are  encapsulated  within  this  Entity  Service  running  on 
the  host  machine.  Applications  wishing  to  use  the  Entity  Service  log  on  as  a  client. 

A  final  word  about  our  choice  of  the  Entity  State  PDU  as  the  primary  avatar  state  vehicle: 
at  the  start  of  this  experiment,  an  evaluation  of  the  human  figure  problem  was  made.  The 
graphical  representation  of  the  human  figure,  in  contrast  to  a  vehicle,  requires  a  relatively  large 
number  of  articulated  parts.  For  example,  the  current  system  supports  fourteen  different  joints  of 
the  human  body  each  with  three  rotations.  In  general,  there  are  two  methods  that  can  be  used  for 
transmitting  human  figures  within  DIS.  An  Entity  State  PDU  with  all  the  articulated  parts  can  be 
sent,  or  a  Data  PDU  can  be  used  to  transmit  the  human  articulated  parts.  Technically,  the  DIS 
standard  does  not  support  articulated  parts  of  human  figure  in  its  articulated  parts  enumeration 
(the  enumeration  focuses  more  on  the  parts  of  vehicles  such  as  a  turret).  However,  we  chose  to 
use  Entity  State  PDUs  because  it  was  felt  that  articulated  part  information  really  should  go  along 
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with  all  the  other  entity  information  even  if  the  DIS  standard  did  not  include  human  figures  in  its 
enumerations  (some  arbitrary  articulated  part  types  were  used). 

Simulation  Startup  from  Browser 

FITT  uses  six  computers,  all  of  which  have  their  own  unique  startup  sequences 
depending  on  which  options  the  user  wishes  to  run.  To  deal  with  experiment  startup 
complications,  we  developed  a  multi-layered  menu  program  in  Java  to  select  the  various  options 
available  and  launch  the  simulation  from  one  terminal. 

The  menu  allows  the  user  to  control  any  part  of  the  simulation  from  one  terminal.  The 
menu  has  options  to  place  the  machines  into  or  take  them  out  of  Multi  Channel  Option  (MCO) 
mode  (allows  for  stereo  view).  It  can  be  used  to  set  up  information  for  data  capture,  start/stop  a 
simulation,  process  captured  data,  play  back  a  previously  mn  simulation,  and  email  the  collected 
data.  It  can  also  bring  the  systems  off  the  network  as  a  restricted  subnet  for  maintaining  a  distinct 
and  stable  experimental  environment. 

The  menu  program  is  a  client/server  combination.  The  server  is  a  Java  application  that 
can  be  run  on  any  one  of  the  computers  used  for  the  simulation.  The  server  receives  commands 
and  options  from  the  client  interface.  Then,  depending  on  the  options,  the  server  uses  the  Java 
System.execO  method  to  remotely  start  up  the  parameters  of  the  simulation  needed  on  the 
various  machines.  After  executing  the  commands,  the  server  updates  the  state  information  of  the 
client. 


Originally,  the  client  interface  was  designed  as  an  applet  that  would  run  through  an 
internet  browser.  It  was  changed  to  an  application  to  get  around  the  security  restrictions  imposed 
by  applets  and  browsers.  Specifically,  the  browser  would  not  allow  an  applet  to  communicate 
with  an  application  unless  the  application  was  running  on  the  same  host  from  which  the  applet 
was  loaded.  Thus,  our  options  were  to  either  set  up  the  machine  mnning  the  application  as  a  web 
server,  or  to  write  the  client  as  an  application  as  well. 

Due  to  the  reboot  process  incurred  while  taking  the  machines  on  and  off  the  lab’s  general 
development  network,  the  menu  needed  to  be  robust.  The  client  and  server  are  both  able  to 
survive  if  the  other  is  killed.  If  the  client  is  killed,  the  server  simply  waits  for  a  new  client  to 
connect  and  then  updates  the  new  client  with  the  current  state  of  the  simulation.  If  the  server  is 
killed  (due  to  system  reboot  while  taking  the  simulation  off  the  network),  the  client  will  display  a 
"Waiting  for  server  message"  and  attempt  to  reconnect  to  the  server.  Once  the  server  is  running 
again,  it  will  connect  to  the  machines  involved,  determine  the  current  state  of  the  simulation,  and 
then  wait  for  the  client  to  reconnect. 

Summary  of  FITT  development  and  Implementation 

In  implementing  FITT,  1ST  built  upon  its  previous  VE  development  efforts  by 
incorporating  a  networked  virtual  environment  system,  a  sophisticated  action-after  critique 
system,  and  a  new  Java-based  menu  system.  The  networked  system  built  upon  previous  work  by 
1ST  (for  the  U.S.  Army  Simulation,  Training  and  Instrumentation  Command  (STRICOM)  under 
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the  Dynamic  Terrain  project)  by  modifying  existing  DIS  protocols  to  support  human  avatars  and 
environment  elements  (for  example,  operable  doors  and  leaking  canisters).  The  action-after 
critique  system  captures  all  environment  network  traffic  and  audio  communication,  and  provides 
a  playback  mechanism,  which  can  be  used  for  reviewing  the  mission  with  the  participants.  The 
new  Java-based  menu  system  replaced  the  old  text-based  menu  system  used  by  previous 
experiments  and  provided  necessary  enhancement  due  to  the  quantity  of  machines  necessary  to 
support  the  experiment  during  run-time.  Overall,  the  FITT  system  provides  a  robust  and  flexible 
networked  virtual  environment  system  using  an  object-oriented  design  running  on  multiple 
workstations. 


Instructional  Strategies  for  Team  Training  in  Virtual  Environments  Experiment 

For  our  first  experiment  looking  at  team  training  in  VE  we  sought  to  implement 
straightforward  (simple)  implementations  of  tradition  instructional  strategies  involving  when  to 
give  guidance  to  the  trainees.  These  were:  guidance  given  either  before  (demonstration),  during 
(coaching),  or  after  (replay)  the  first  practice  mission,  or  not  at  all  (control  group).  Thus,  the 
instructional  strategies  differ  in  when  guidance  is  given.  We  did  not  expect  that  any  particular 
strategy  would  be  significantly  superior  to  the  others,  but  rather  wished  to  conduct  an 
exploratory  comparison  of  their  effectiveness. 

Instructional  Strategies 


Demonstration 


Demonstrations  provide  an  observer  with  information  about  the  required  behaviors, 
actions,  or  strategies  associated  with  a  task.  We  believe  that  the  unique  capabilities  of  VE  can 
enhance  what  has  already  proven  to  be  an  effective  instructional  strategy. 

In  the  demonstration  group,  the  team  members  watched  a  replay  of  a  mission  conducted 
by  an  “expert”  team  highly  practiced  with  the  tasks.  The  replay  is  of  the  movement  of  the 
avatars,  viewed  from  above  and  centered  on  the  team  leader,  as  they  correctly  perform  the 
mission  tasks.  Thus,  the  trainees  see  the  avatars  in  proper  formation,  and  hear  examples  of 
appropriate  radio  cormnunications,  before  they  attempt  the  practice  and  test  missions.  The 
demonstration  provides  an  example  of  good  team  coordination  and  a  feel  for  the  tempo 
appropriate  for  the  individual  tasks  and  the  overall  mission. 

The  demonstration  mission  was  produced  by  having  an  experienced  team  perform  a 
mission,  and  videotaping  a  playback  of  that  mission.  (Prior  experience  was  based  on  performing 
missions  to  help  test  the  system  and  several  missions  conducted  specifically  to  rehearse  for  the 
demonstration  mission.)  During  the  playback,  the  view  was  zoomed  in  and  out.  For  example, 
when  the  team  assumed  the  “stack”  formation  the  view  was  zoomed  in  to  give  a  better  view  of 
the  proper  formation.  During  engagements  with  the  enemy,  the  view  was  zoomed  in  and  out  so 
that  both  the  team  and  Opposing  Forces  were  shown.  The  demonstration  team  featured  three 
easily  distinguished  voices:  mature  man  (Team  Leader)  youthful  man  (Equipment  Specialist), 
mature  woman  (Mission  Commander).  There  was  no  narration  additional  to  communication  that 
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occurred  as  part  of  the  mission. 
Coaching 


Coaching  involves  knowing  what  to  say,  and  when  to  say  it,  to  the  trainees.  In  the 
coaching  condition,  the  participants  received  guidance  from  the  mission  commander  while  they 
attempted  the  missions.  Coaching  provided  immediate  feedback  and  prevented  the  participants 
from  practicing  procedures  incorrectly.  The  frequency  of  coaching  was  expected  to  naturdly 
“fade”  as  the  participants  required  less  guidance  as  they  gained  experience.  A  potential  problem 
with  coaching  is  that  it  may  rapidly  bring  a  team  to  a  high  level  of  performance,  but  may  also 
lead  to  problems  when  the  coaching  “crutch”  is  not  available  during  the  test  mission. 

Replay  (with  critique) 

After  performing  a  mission,  the  participants  watched  a  replay  of  that  mission.  During  the 
replay,  the  mission  commander  pointed  out  the  strengths  and  weaknesses  of  the  mission.  This 
approach  is  similar  to  the  after-action  review  (AAR).  The  AAR  is  the  Army’s  approved  method 
for  providing  feedback  to  trainees  as  part  of  their  performance  oriented  training  (Meliza,  1996). 
The  AAR  is  a  professional  discussion  conducted  after  training  exercises  to  maximize  and 
reinforce  learning  by  involving  the  participants  in  the  training  diagnostic  process.  An  effective 
AAR  is  one  in  which  performance  problems  are  identified,  defined,  and  solved  in  such  a  manner 
that  allows  the  trainees  to  learn  from  their  performance.  The  main  goal  is  to  identify  strengths 
and  weaknesses  while  focusing  on  problem  solving  to  allow  trainees  to  determine  their  level  of 
skill  and  decide  what  they  need  to  do  to  improve  their  performance.  Through  participation  in  an 
AAR,  trainees  get  to  identify  the  tasks  accomplished  and  the  tasks  requiring  improvement. 

Among  the  ways  that  our  after-action  critique  conducted  in  this  experiment  differed  from 
a  standard  Army  AAR  was  that  the  replay  was  not  paused  for  comments.  In  addition,  discussion 
by  the  participants  was  greatly  limited. 

In  both  the  coaching  and  replay  conditions,  the  experimenter  had  a  checklist  to  aid  in 
scoring  team  performance  and  providing  consistent  feedback.  The  checklist  items  were  those 
included  in  the  mission  scoring  criteria  (Table  2). 

No  external  feedback  (control  group) 

This  group  received  no  feedback  other  than  that  intrinsic  to  performing  the  missions,  and 
provided  a  baseline  with  which  to  compare  the  performance  of  the  instructional  strategy  groups. 
In  some  experiments  the  control  group  represents  a  “no  training”  condition  to  determine  chance 
performance.  This  is  not  the  case  in  our  design.  The  control  group  teams  studied  the  team 
mission  training  manual  as  did  the  instructional  strategy  groups,  and  completed  two  practice 
missions  before  conducting  a  third  test  mission. 

Note  that  total  training  time  was  held  constant  across  groups.  The  control  group  teams, 
therefore,  performed  one  more  practice  mission  than  the  demonstration  and  replay  teams.  The 
demonstration  and  replay  teams  watched  a  mission  replay  in  place  of  a  second  practice  mission. 
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Experimental  Procedure  and  Desi 


Ninety-four  participants  were  recruited  from  local  colleges  and  scheduled  two  at  a  time. 
Participants  had  the  option  of  receiving  extra  credit  points  from  their  instructors  or  seven  dollars 
for  each  hour  of  participation.  If  one  of  the  scheduled  two  participants  did  not  show  up,  then  an 
experimenter’s  confederate  served  as  the  second  team  member.  Participants  read  and  signed  a 
consent  form,  then  completed  background  information  and  baseline  (pre-immersion)  Simulator 
Sickness  Questionnaires  (SSQ).  (See  Kennedy,  Lane,  Lilienthal,  Berbaum,  and  Hettinger  (1992) 
for  a  comprehensive  description  of  the  SSQ.)  Next,  they  watched  a  videotape  showing  how  to 
walk  and  use  equipment  in  the  VE.  A  coin  flip  determined  who  would  be  the  Team  Leader.  (For 
teams  with  an  experimental  confederate,  the  participant  was  always  designated  the  Team  Leader. 
A  list  of  rules  defined  the  limits  on  the  ways  the  confederate  could  help  the  leader.  Basically,  the 
confederate  performed  the  Equipment  Specialist  duties  very  efficiently,  but  was  instructed  not  to 
help  the  leader  with  navigating  the  building,  performing  leader  procedures,  or  monitoring  or 
dealing  with  the  time  limit.) 

Two  different  VEs  provided  familiarization  with  using  the  FITT  system.  The  first  VE 
provided  practice  walking  in  VEs.  After  a  break,  a  second  VE  allowed  the  participant  to  practice 
using  equipment,  more  demanding  walking  tasks,  and  to  recognize  a  team  mate,  types  of  enemy, 
and  innocent  bystanders.  Thus,  before  participants  conducted  a  mission  as  a  team,  they  had  each 
practiced  individually  in  moving  through  VEs,  aiming  and  firing  the  pistol,  and  operating  the 
equipment  unique  to  their  assigned  role. 

Next,  the  participants  read  the  mission  training  manual.  A  paper  and  pencil  knowledge 
test  based  on  the  mission  procedures  was  then  administered.  The  participants  were  told  which,  if 
any,  items  they  missed  and  the  correct  responses. 

Up  to  this  point,  all  participants  received  the  same  treatment.  They  were  then  assigned  to 
one  of  the  four  conditions  shown  in  Table  1. 

For  purposes  of  illustration  of  the  training  phase  we  consider  the  control  group  first.  The 
teams  in  this  group  conducted  three  missions,  with  a  rest  break  between  missions.  Each  of  these 
missions  was  different  but  of  equivalent  difficulty  as  defined  by  the  number  of  canisters  and 
OPFOR  to  be  dealt  with.  The  mission  scores  for  the  control  group  allow  us  to  plot  change  in 
performance  as  a  function  of  practice  without  external  feedback  or  guidance.  Performance  on  the 
third  mission  can  be  compared  with  the  test  mission  of  the  other  instructional  strategy  groups. 

The  demonstration  group  watched  a  replay  of  a  mission  performed  by  a  team  highly 
familiar  and  practiced  with  the  mission  tasks.  The  demonstration  was  given  in  place  of  one 
practice  mission.  After  watching  the  demonstration,  the  team  then  had  a  practice  mission  session, 
and  then  the  test  mission  session.  In  a  similar  manner,  in  place  of  a  second  practice  mission 
session,  the  replay  group  teams  watched  a  replay  of  their  own  performance  of  the  first  practice 
mission.  In  the  coaching  condition,  the  commander/experimenter  provided  prompts  or 
suggestions  as  the  team  conducted  the  mission. 


Table  1. 


Research  Design 


Group 

Trainins  Phase 

Test 

(Mission  #  ll 

(Mission  #  2) 

(Mission  #  3) 

Demonstration 

Watch  Demonstration 

Practice 

Test 

Coaching 

Practice  with  Coaching 

Practice  with  Coaching 

Test 

Replay 

Practice 

Critique  During  Replay 

Test 

Control 

Practice 

Practice 

Test 

The  demonstration  and  each  mission  exercise,  replay,  and  test  were  eight  minutes  in 
duration.  This  time  limit  made  sure  that  all  groups  had  the  same  total  training  time.  Two  different 
scenarios  were  counterbalanced  across  the  first  and  third  missions.The  scenarios  differed  in  the 
layout  of  the  rooms  within  the  building  and  the  placement  of  canisters  and  OPFOR. 

After  each  mission,  the  participants  completed  another  SSQ.  After  the  last  mission, 
participants  were  told  the  purpose  of  the  experiment  and  given  the  opportunity  to  ask  questions. 
Total  time  was  two  hours. 

Performance  Measures 


The  criteria  for  measuring  team  performance  of  the  mission  procedures  are  presented  in 
Table  2.  Data  streams  from  each  mission  were  replayed  and  recorded  on  videotape.  Raters  used 
checklists  to  score  each  of  the  procedures,  pausing  or  replaying  mission  segments  when 
necessary  to  aid  scoring. 
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Table  2 


Description  of  Mission  Procedures  and  Scoring  Criteria 


Procedure 

Description  (scoring  criteria) 

Room  search  order 

Is  the  right  turn  rule  observed  by  both  team  members  in  the  search  of 
building’s  rooms? 

Proper  entry  formation 
and  coordination 

Do  team  members  align  themselves  properly  with  the  doorway,  and 
recite  "stack/ready/go"  protocol  correctly? 

Room  entry 

Does  the  team  leader  move  far  enough  into  the  room  to  allow 
equipment  specialist  to  pass  through  the  doorway? 

Effectiveness  of  search 

Does  team  penetrate  the  room  adequately  to  see  around  all 
obscuring  furniture,  etc;  were  any  canisters  missed? 

Room  exit 

Does  team  exit  the  room  together,  with  team  leader  in  front? 

Room  marking 

Does  team  leader  use  paint  gun  to  mark  the  outside  of  the  room 
(above  the  doorway)that  was  just  searched? 

Pistol  in  hand 

Does  the  equipment  specialist  have  pistol  selected  when  leaving 
room? 

"Contact"  announced 

Do  team  members  clearly  announce  "contact"  upon  encountering 
enemy? 

Appropriate  target, 
number  of  shots,  and 

clear  line  of  fire 

Do  team  members  incapacitate  enemy  (and  not  innocent  bystanders; 
do  they  fire  a  minimum  of  tranquilizer  darts,  and;  do  they  move 
around  each  other  (and  other  obstructions)  to  obtain  a  clear  line  of 
fire? 

Reports  contact 

Does  team  leader  report  to  the  commander  that  contact  with  enemy 
has  occurred? 

Correct  call  sign;  proper 
format 

In  reporting  the  above  information,  does  the  team  leader  initiate  and 
carry  out  commo  in  accordance  with  protocols  in  the  training 
manual? 

Correct  enemy  type  and 
number 

In  reporting  to  the  commander,  does  the  team  leader  properly 
identify  the  enemy  type  (looter  or  terrorist)  and  number? 

Hazardous  gas  canisters 

Does  the  equipment  specialist  check  canister(s),  if  present;  does  the 
equipment  specialist  then  inform  the  team  leader  of  the  canister's 
original  and  current  status  (e.g.,  "active  and  now  deactivated;"  or 
"dud")? 

Reporting  of  hazardous 
gas  canisters 

Does  the  team  leader  properly  describe  to  the  commander  both  the 
number  and  status  of  the  canisters  encountered,  and  follow  correct 
call  sign  and  format  protocols? 
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Results 


This  section  begins  with  background  information  on  the  research  participants.  Next, 
performance  measures  are  presented.  There  are  two  categories  of  performance  measures.  The 
first  consists  of  those  measures  recorded  once  per  mission.  (For  example,  did  the  team  exit  the 
building  within  the  time  limit?)  The  second  category  consists  of  measures  taken  several  times  per 
mission,  such  as  proper  formation  for  entering  a  room  and  actions  on  encountering  enemy. 
Following  the  logic  of  the  experimental  design,  performance  on  the  last  mission  is  the  primary 
focus  of  the  summaries  of  both  categories  of  performance  measures.  Performance  on  the  practice 
missions  is  also  described.  Performance  by  teams  with  two  participants  is  compared  with  that  of 
teams  with  a  confederate.  Frequency  and  severity  of  the  occurrence  of  simulator  sickness  are 
discussed  briefly.  Participants’  ratings  of  several  characteristics  and  dimensions  of  teamwork  are 
presented  and  ratings  of  aspects  of  presence,  the  sense  of  being  immersed  in  the  VE,  are 
summarized. 

Participant  background  information 

Ninety-four  participants  were  involved  in  this  experiment.  Fifty-two  (55%)  were  women 
and  42  (45%)  were  men.  Their  age  ranged  from  17  to  54  with  an  average  of  21.  Nine  of  them 
withdrew  because  of  simulator  sickness.  Three  teammates  of  those  simulator  sickness 
withdrawals  could  not  be  accommodated  with  teammates  and  are  not  included  in  the  participant 
count  for  determining  the  number  and  percentage  of  participants  who  withdrew  because  of 
simulator  sickness.  Although  they  did  not  withdraw,  they  also,  through  no  fault  of  their  own,  did 
not  complete  the  experiment.  Obviously,  performance  data  are  not  available  for  those  teams  that 
did  not  complete  the  experiment.  In  addition,  for  some  of  the  teams  (involving  ten  participants) 
that  did  complete  the  experiment,  data  capture  and  storage  problems  resulted  in  loss  of  some 
performance  data.  Therefore,  the  number  of  participants  varies,  as  indicated,  across  the  various 
analyses. 

The  remaining  8 1  participants  formed  49  teams,  26  “real”  teams  in  which  both  members 
were  experimental  subjects  and  23  confederate  teams  in  which  the  confederate  performed  the 
role  of  Equipment  Specialist.  The  percentage  of  real  teams  in  the  Control,  Demonstration, 
Coaching,  and  Replay  groups  were  55%,  50%,  67%  and  43%,  respectively. 

Performance  data 

Exiting  the  building  within  the  time  limit.  Figure  6  presents  the  percentage  of  teams  per 
group  that  exited  within  the  time  limit  or  at  least  attempted  to  exit.  The  number  of  teams  with 
usable  data  per  condition  were:  control  =11,  demonstration  =  12,  coaching  =  12,  and  replay  = 

14.  Equivalent  procedures  were  used  for  the  first  mission  for  the  replay  and  control  groups, 
therefore  data  for  those  groups  were  combined  as  “baseline”.  Baseline  represents  the 
performance  on  the  first  mission  without  a  prior  demonstration  or  coaching  during  the  mission. 
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E3  Exited  in  time  B  Attempted  to  exit 
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Figure  6.  Percentage  of  teams  per  group  that  exited  within  the  time  limit  or  at  least  attempted  to 
exit.  Baseline  represents  the  first  mission  for  the  control  group  and  replay  group  combined. 


Because  of  the  small  number  of  teams,  we  used  the  odds  ratio  approach  for  comparing 
group  performance.  The  ratio,  calculated  by  dividing  the  odds  of  one  event  by  another,  provides 
results  in  the  form  of  a  direct  comparison  between  variables  (e.g.,  "a  team  in  the  demonstration 
group  is  X  times  as  likely  to  successfully  exit  as  is  a  team  in  the  control  group").  Advantages  of 
the  odds  ratio  include  its  being  unaffected  by  sample  size  or  by  unequal  row  or  column  totals 
(Howell,  1997).  Stated  in  terms  of  an  odds  ratio,  teams  from  all  groups  combined  were  about 
13.8  times  more  likely  to  exit  in  time  during  the  test  mission  than  the  baseline  teams  were  in  the 
first  practice  mission  (chi-square  (1,  N  =  74)  =  16.27,  p  <  .001).  Thus  the  training  sessions 
improved  performance  on  this  performance  measure. 

The  question  of  whether  teams  improve  with  practice  alone  was  addressed  by  comparing 
the  number  of  control  group  teams  that  successfully  exited  during  the  first  mission  with  the 
number  that  exited  during  the  test  mission.  Following  the  analysis  described  by  Siegel  (1956)  for 
related  samples  (control  team  is  used  as  its  own  control)  and  a  small  (less  than  5  )  frequency  for 
one  cell  of  the  contingency  table,  a  binomial  test  yielded  p  =  .016. 

Thus  the  performance  of  both  control  and  instructional  strategy  teams  improved  from  the 
first  to  the  third  mission.  An  instructional  strategy  team  was  about  2.7  times  more  likely  to  exit 
in  time  than  was  a  control-group  team.  However,  following  the  analysis  described  by  Siegel 
(1956)  for  independent  samples  and  a  small  frequency  for  one  cell  of  the  contingency  table,  a 
Fisher  exact  probability  test  yielded  p  =  .14  for  the  same  or  a  stronger  association.  Thus  the 
apparently  superior  performance  of  the  instructional  strategy  teams  relative  to  control  group 
teams  was  not  statistically  significant. 


During  data  collection,  we  noticed  that  some  teams  that  failed  to  exit  in  time  attempted  to 
exit  but  failed,  whereas  others  did  not  even  try  to  exit.  Teams  could  be  so  caught  up  in  the 
mission,  particularly  during  encounters  with  enemy,  that  they  forgot  to  monitor  their  air  gauges. 
For  teams  that  attempted  to  exit  but  failed;  some  waited  until  too  late  to  start  back,  some  started 
back  with  enough  time  but  then  decided  “Maybe  we  have  time  to  search  one  more  room.”  Teams 
could  also  become  lost  in  attempting  to  retrace  their  paths.  It  was  generally  very  clear  when  a 
team  decided  to  exit  in  that  usually  the  Team  Leader,  but  occasionally  the  Equipment  Specialist, 
would  express  something  to  the  effect  that  “We  need  to  leave  now.” 

Figure  6  presents  the  percentages  of  teams  who  at  least  attempted  to  exit.  All  teams 
combined  on  the  third  mission  were  3.06  times  more  likely  to  attempt  to  exit  than  were  baseline 
teams  (chi-square  (1,  N=74)  =  14.55  ,  p  <.001.  Looking  at  the  effect  of  practice  alone,  the  control 
group  teams  were  about  3.8  times  as  likely  to  attempt  to  exit  during  the  test  mission  than  the 
baseline  teams  during  their  first  mission.  A  binomial  test  comparing  the  number  of  control  group 
teams  that  attempted  to  exit  during  the  first  mission  with  the  number  that  attempted  to  exit  during 
the  test  mission  yielded  a  significant  p  =  .016. 

For  the  test  mission,  an  instmctional  strategy  group  team  was  3.12  times  more  likely  to 
attempt  to  exit  than  a  control  group  team.  However,  this  difference  was  not  statistically 
significant  (chi-square  (1,  N  =  49)  =  1.5,  p  =  .22). 

Procedural  tasks 


A  single  composite  score  was  generated  for  each  mission  indicating  the  percentage  of 
correctly  executed  tasks  based  on  the  total  number  of  appropriate  opportunities  to  perform  the 
tasks.  Percentage  of  procedures  performed  correctly,  and  the  number  of  rooms  entered,  out  of 
total  possible  for  the  first  and  test  missions  are  shown  in  Figure  7.  (For  the  percentage  of 
procedures  performed  correctly,  the  number  of  teams  with  usable  data  per  condition  were: 
control  =  7,  demonstration  =  9,  coaching  =11,  and  replay  =  14.)  In  a  general  sense,  the 
percentage  score  represents  the  accuracy  with  which  the  teams  perform  in  contrast  to  the  number 
of  rooms  searched  which  represents  the  speed  of  the  building  search.  For  both  of  these  measures 
Analyses  of  Variance  for  a  practice  effect  (control  group  mission  1  versus  mission  3)  and 
instructional  strategy  effect  (test  mission,  the  3  strategies  versus  control)  were  not  statistically 
significant. 

Examination  of  the  scores  of  individual  teams  indicated  a  wide  range  in  performance. 

The  highest  percentage  of  procedures  score  for  the  last  mission  was  97.  That  indicates  for 
practically  every  room  entered  and  searched,  and  every  encounter  with  OPFOR  and  canisters, 
that  team  correctly  performed  all  of  the  tasks  listed  in  Table  2.  That  team  completed  searching 
seven  rooms  and  returned  to  the  designated  area  with  13  seconds  left  on  the  countdown  timer. 
The  lowest  score  was  34.  That  team  entered  8  rooms,  correctly  executed  only  34  percent  of  the 
tasks,  and  did  not  attempt  to  exit  within  the  time  limit. 
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Figure  7.  Percentage  of  procedures  performed  correctly  and  number  of  rooms  entered  out  of‘ 
total  possible  for  the  first  and  test  missions.  Baseline  represents  the  first  mission  for  the  control 
group  and  replay  group  combined. 

Performance  as  a  function  of  team  composition 

Table  3  presents  a  comparison  of  performance  of  teams  made  up  of  two  recruited 
participants  with  that  of  teams  composed  of  a  participant  serving  as  the  team  leader  and  an 
experimenter’s  confederate  as  the  equipment  specialist.  For  number  of  rooms  searched  and 
percent  of  correct  procedures,  t-tests  for  independent  samples  did  not  approach  statistical 
significance. 


Table  3 

Test  Mission  Performance  as  a  Function  of  Team  Composition:  Two  Participants  versus  a 
Participant  and  an  Experimenter’s  Confederate 


Performance  Measure 

Equipment  Specialist 

Participant 

Confederate 

%  exited  in  time 

52  (23) 

%  attempted  to  exit 

78  (23) 

Mean  number  of  rooms  searched 

8.2  (23) 

Mean  percent  correct  procedures 

78  (20) 

78  (21) 

Note:  The  total  number  of  teams  is  shown  in  parentheses. 
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Videotapes  of  23  test  missions  conducted  by  teams  including  a  confederate  were 
reviewed  to  determine  if  the  confederate  followed  the  directions  for  role  playing.  The 
confederate  was  instructed  to  serve  efficiently  as  the  Equipment  Specialist  but  to  provide  no 
information,  beyond  what  was  called  for  in  the  training  manual,  to  influence  the  performance  of 
the  Team  Leader.  In  four  of  the  23  missions  the  confederate  was  judged  to  be  “too  helpful”. 

Simulator  Sickness 


About  9%  of  the  participants  withdrew  from  the  experiment  because  of  simulator 
sickness.  All  of  the  participants  who  withdrew  because  of  simulator  sickness  were  women. 
Reason  and  Brand  (1975)  suggest  that  women  tend  to  be  more  susceptible  to  motion  sickness 
than  men.  Given  that  each  mission  was  relatively  brief,  only  about  eight  minutes  in  the  HMD  at  a 
time,  and  rest  breaks  were  given  between  missions,  the  attrition  rate  indicates  that  simulator 
sickness  is  still  a  challenge  that  must  be  considered  in  developing  practical  training  applications 
using  immersive  VEs. 

Fatigue  was  the  most  frequently  reported  symptom  on  the  pre-immersion  SSQ.  For  all 
five  post-immersion  SSQs,  eyestrain  was  the  most  frequently  reported  symptom.  Only  on  the 
SSQ  administered  after  the  last  mission  were  any  symptoms  rated  as  severe:  dizzy  eyes  open  and 
stomach  awareness.  The  last  mission  also  had  the  greatest  variety  of  symptoms  reported  with 
each  of  the  16  symptoms  reported  by  at  least  two  participants. 

For  those  participants  who  completed  the  experiment,  the  mean  Total  Severity  score  for 
the  SSQ  administered  after  the  test  mission  was  9  and  the  standard  deviation  was  17.  For  those 
who  withdrew,  the  mean  was  38  and  the  standard  deviation  was  20  for  the  SSQ  administered 
after  the  last  mission  they  attempted.  Addition  information  on  simulator  sickness  data  from  this 
experiment  is  presented  in  Lampton,  Rodriguez,  and  Cotton  (2000). 

Participant  Ratings  of  Team  Processes 

Because  there  was  not  enough  time  to  administer  lengthy  questionnaires  during  the 
experimental  session,  we  asked  participants  to  take  home  the  questionnaires  and  return  them 
within  one  week.  The  questionnaires  addressed  team  processes  and  “presence”  the  extent  to 
which  participants  feel  immersed  in  the  VE.  This  take  home  approach  was  only  attempted  for 
about  the  last  third  of  the  data  collection  effort.  Forty-two  of  the  participants  were  given  a  set  of 
questionnaires  to  take  with  them  and  return  them  within  one  week.  Forty  did  so. 

Two  instruments  were  used  to  measure  participants’  ratings  of  team  processes  and 
satisfaction.  The  first  questionnaire,  adapted  from  Harvey  and  Drolet,  (1994),  consisted  of  a  list 
of  14  team  characteristics:  cooperation,  communication,  mutual  support,  mutual  respect, 
atmosphere,  cohesion,  pride,  trast,  leadership,  participation,  decision  making,  goals  and  roles, 
problem  solving,  and  climate.  Participants  rated  each  characteristic  with  a  scale  ranging  from 
“none”  to  “excellent”.  Table  4  presents  the  mean  value  and  rating  frequencies  for  each  item.  The 
table  lists  the  items  in  descending  order  of  the  mean  values.  Note  that  most  of  the  ratings  are 
favorable.  Two  participants  indicated  that  the  process  of  “pride”  was  not  applicable  to  their  team 
membership. 
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Table  4 


Mean  Values  and  Rating  Frequencies  for  Team  Process  Items 


Item  (mean) 

Good 

Fair 

Poor 

None 

Participation 

56.4 

38.5 

55.3 

36.8 

7.9 

Trust  (3.41) 

46.2 

48.7 

3.26 

41 

Kligl 

Goals  and  Roles 

3.23 

41 

41 

17.9 

Climate) 

3.18 

33.3 

51.3 

15.4 

Cohesion 

3.15 

33.3 

17.9 

3.15 

33.3 

17.9 

Pride** 

3.14 

32.4 

48.6 

18.9 

3.13 

36.8 

39.5 

23.7 

3.13 

35.9 

42 

23.1 

Decision  Making 

38.5 

30.8 

30.8 

Leadership 

41 

28.2 

2.6 

Communication 

2.95 

30.8 

41 

20.5 

7.7 

Note:  Means  based  on  Excellent  =  4,  Good  =  3,  Fair  =  2,  Poor  =  1,  None=  0 
Percentages  based  on  N  =  39 
*n=38 
**n  =  37 


The  second  questionnaire,  adapted  from  Eitington  (1996),  consisted  of  eight  statements 
about  the  quality  of  the  team  experience  (roles  are  clear,  roles  are  balanced,  decision  making  is 
effective,  team  members  work  well  together,  I  need  to  work  with  the  other  member  to  get  my  job 
done.  I’m  satisfied  with  my  level  of  participation  in  this  team,  conflict  is  handled  appropriately 
and  effectively).  Rating  choices  could  vary  from  “strongly  disagree”  to  “strongly  agree”.  Table  5 
presents  the  mean  rating  value  and  rating  percentages  for  each  item.  The  table  lists  the  items  in 
descending  order  of  the  mean  values.  Again,  in  general  the  participants  reported  a  high  level  of 
satisfaction  with  their  membership  on  the  team.  The  dimension  with  the  highest  mean  score  was 
'roles  are  clear’  with  a  3.38.  The  lowest  scoring  characteristic  was  'roles  are  balanced’  with  a 
mean  of  2.85.  Clearly,  the  roles  were  not  and  were  not  intended  to  be  balanced  in  that  the  Team 
Leader  was  expected  to  lead  the  mission. 
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Table  5 


Mean  Values  and  Percentages  for  Team  Process  Ratings. 


Item 

Mean 

Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 

Roles  are  clear. 

3.38 

46.2% 

46.2 

7.7 

Team  members  work  well 
together. 

3.36 

41 

53.8 

5.1 

I’m  satisfied  with  my 
membership  in  this  team. 

3.33 

43.6 

48.7 

5.1 

2.6 

I  need  to  work  with  the  other 
member  to  get  my  job  done. 

3.28 

64.1 

17.9 

7.7 

2.6 

7.7 

Conflict  is  handled 
appropriately  and  effectively. 

3.23 

35.9 

53.8 

7.7 

2.6 

I’m  satisfied  with  my  level  of 
participation  in  this  team. 

3.18 

35.9 

51.3 

7.7 

5.1 

Decision  making  is  effective. 

3.08 

28.2 

51.3 

20.5 

Roles  are  balanced. 

2.85 

28.2 

43.6 

12.8 

15.4 

Note:  Means  based  on  Strongly  Agree  =  4,  Agree  =  3,  Neutral  =  2,  Disagree  =  1,  Strongly 
Disagree  =  0 

Percentages  based  on  N  =  39 


Presence 


Previous  ARI VE  research  has  employed  the  Presence  Questionnaire  (PQ)  and  Immersive 
Tendencies  Questionnaire  (ITQ)  developed  by  Witmer  and  Singer  (1994).  The  PQ  was  designed 
to  measure  the  extent  to  which  the  VE  participant  is  inunersed  and  involved  with  the  VE.  They 
define  presence  as  the  subjective  experience  of  being  in  one  place  or  environment  (the  VE)  when 
one  is  physically  situated  in  another  (the  real-world  VE  system).  The  ITQ  was  developed  to 
measure  the  tendency  to  become  involved  or  immersed.  Both  questionnaires  use  participant  self- 
report. 


Forty  participants  completed  the  questionnaires.  We  had  expected  that  the  mean  for  the 
PQ  Total  Score  would  be  higher  than  previous  ARI  VE  experiments.  This  expectation  was  based 
upon  the  use  in  this  experiment  of  VEs  and  tasks  that  are  more  complex,  the  presence  of  a 
teammate,  and  the  use  of  sound  effects.  However,  the  PQ  Total  Score  mean  and  standard 
deviation  for  this  experiment  (M  =  98.54,  SD  =  13.48)  were  similar  to  averaged  values  for 
previous  experiments  (M  =  98.1 1,  SD  =  15.78)  reported  in  Witmer  and  Singer  (1994).  Also 
contrary  to  our  expectations,  the  correlation  between  PQ  Total  and  ITQ  Total  (r=  .164,  p=  .340) 
was  not  significant. 
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Discussion 


The  FIT  T  System 

This  was  the  first  experiment  we  conducted  with  FITT.  In  addition  to  examining  the  use 
of  instructional  strategies  in  distributed  immersive  VEs,  this  research  also  served  as  a 
demonstration  of  new  immersive  VE  technologies  and  as  a  usability  test  of  new  approaches  to 
moving  in,  and  interacting  with,  immersive  VEs. 

In  general,  the  FITT  seems  to  function  well  as  a  system  for  conducting  research  on  team 
training  using  immersive  VEs.  The  mission  tasks  combined  many  generalizable  task  components 
across  related  but  separate  domains,  such  as  infantry  tactics,  SWAT  tactics  and  HAZMAT 
operations.  The  HAZMAT  equipment  and  environment  that  support  the  mission  provided  a 
concise  and  logical  storyline.  In  addition,  the  avatars’ HAZMAT  gear  lead  to  user  acceptance  of 
system  limitations. 

The  cognitive  aspects  of  the  mission,  and  not  the  mechanics  of  using  the  VE  interface, 
were  the  greatest  challenge  for  most  participants.  Most  participants  reported  that  they  enjoyed 
the  experience,  and  seemed  highly  motivated  to  perform  well  on  the  missions. 

A  few  individuals  seemed  overwhelmed  by  the  experience,  perhaps  by  the  immersive  VE 
and/or  the  stress  of  leading  a  team,  and  did  not  function  effectively  as  team  leaders  even  after 
three  practice  missions.  From  a  training  research  perspective  this  is  not  necessarily  a  bad  thing, 
in  that  some  individuals  would  struggle  if  placed  in  a  leadership  position  in  comparable  real- 
world  tasks. 

The  FITT  system  for  walking  in  the  VE  worked  very  well.  The  locomotion  system  has 
two  strong  points:  the  participant’s  vestibular  and  ocular  systems  are  always  in  accord,  and  the 
system  requires  little  train-up  time  for  a  novice. 

The  structured  approach  to  train-up  on  the  FITT  system  was  effective  and  should  be 
considered  for  other  VE  training  applications.  The  narrated  videotape  provided  the  participants 
with  a  good  overview  of  the  system  before  they  were  immersed.  Simplified  VEs  and  tasks 
designed  specifically  to  familiarize  the  participants  with  the  FITT  system  avoided 
inconsistencies  that  can  result  from  a  “free  play  “  familiarization  session. 

In  general,  the  FITT  system  operated  reliably  over  a  data  collection  period  spanning 
several  months.  The  few  significant  problems,  described  below,  have  been  flagged  for  correction 
in  subsequent  versions  of  the  FITT  system. 

In  our  experimental  design,  an  important  measure  of  performance  was  to  be  the  number 
of  times  the  OPFOR  shot  members  of  a  team  per  encounter.  A  team  that  followed  procedures 
should  be  hit  fewer  times  than  a  team  that  did  not  follow  procedures.  For  example,  following 
appropriate  procedures,  a  good  team  would  enter  the  room  quickly  with  the  Team  Leader 
moving  far  enough  into  the  room  to  allow  the  Equipment  Specialist  to  enter.  The  Team  Leader 
would  cover  the  left  side  of  the  room,  the  Equipment  Specialist  the  right  side.  Both  team 
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members  would  announce  contact,  specifying  the  location  and  number  of  OPFOR  sighted,  and 
rapidly  engage  the  enemy.  The  team  would  sustain  few  hits  even  if  caught  in  a  crossfire.  In 
contrast,  if  a  Team  Leader  froze  in  the  doorway  or  either  member  failed  to  cover  their  assigned 
sector,  that  team  would  sustain  many  hits.  For  various  reasons  related  to  network  speed  and 
object  modeling,  correctly  aimed  shots  by  team  members  did  not  always  result  in  timely 
incapacitation  of  the  OPFOR.  This  resulted  in  several  instances  in  which  teams  quickly  and 
accurately  performed  action  on  contact  procedures,  but  still  were  hit  many  times  by  the  OPFOR. 
This  unwanted  delay  varied  considerably  across  missions  and  therefore  the  number  of  hits 
received  was  not  a  usable  measure  of  team  performance.  As  might  be  expected,  a  related 
problem  was  that  participants  would  continue  to  fire  at  OPFOR  until  the  OPFOR  would  fall. 
Therefore,  the  shots  fired  per  encounter  measure  was  more  a  function  of  system  delay  than  of 
aiming  accuracy  and  therefore  was  also  unusable. 

For  a  few  data  collection  sessions,  data  capture  and  storage  were  corrupted  to  the  extent 
that  data  from  those  sessions  were  unusable.  For  some  other  sessions,  problems  with  audio  data 
capture  prevented  analysis  of  the  communication  procedures.  In  addition  to  correcting  data 
capture  reliability,  the  system  should  provide  some  indication  to  the  experimenter  that  data 
capture  is,  or  is  not,  occurring. 

A  major  problem  we  encountered  in  this  research  was  unrelated  to  the  FITT  system  itself. 
In  designing  and  conducting  research  on  training  applications  of  VE  there  are  challenges 
associated  with  the  new  and  very  rapidly  changing  software  and  hardware  technologies 
necessary  for  the  creation  of  immersive,  interactive  VEs.  However,  in  conducting  this 
experiment,  the  greatest  challenge  by  far  was  the  recruitment  of  participants.  There  were  two 
aspects  to  this;  participant  “no-shows”,  and  the  great  variation  across  participants  along 
characteristics  that  we  might  expect  to  influence  mission  performance. 

Our  design  called  for  two  participants  at  a  time.  In  over  seven  years  of  VE  research 
conducted  in  the  ARI/IST  laboratory,  we  have  experienced  systematic  problems  with  “no 
shows”,  that  is,  students  who  agree  to  participate  but  fail  to  report  at  the  agreed  upon  time.  For 
this,  our  first  team  training  experiment,  no-shows  were  especially  problematic  in  that  our  design 
called  for  two  participants  at  a  time.  As  explained  in  detail  in  the  procedure  section,  when  only 
one  of  the  two  participants  scheduled  reported  for  the  experiment,  we  employed  an 
experimenter’s  confederate  to  serve  as  the  second  team  member.  The  use  of  a  confederate, 
holding  constant  the  capabilities  and  motivation  of  the  second  team  member,  could  be  a  useful 
approach  to  team  training  research  and  perhaps  even  team  training  for  that  matter.  In  addition, 
confederates  could  be  useful  in  developing  and  testing  role-playing  rules  to  be  implemented  in 
computer-generated  friendly  forces.  However,  in  this  situation  the  use  of  a  confederate  was  by 
necessity,  not  by  choice,  and  added  the  complication  of  trying  to  balance  across  the  four 
instmctional  strategy  groups  the  number  of  teams  composed  of  two  recruited  participants  versus 
one  recruited  participant  and  a  confederate. 

A  second  problem  was  that  participants  varied  greatly  in  characteristics  that  we  might 
expect  to  influence  performance  on  mission  procedures.  Some  of  our  participants  had  prior 
experience  that  probably  helped  them  during  the  missions.  For  example,  our  participants 
included  a  veteran  SWAT  team  leader.  Reserve  Officer  Training  Corps  (ROTC)  cadets,  and  avid 
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computer  gamers.  Other  participants  had  almost  no  experience  with  computers,  and  for  some 
English  was  not  their  native  language. 

Both  of  these  problems  could  be  mitigated  by  a  formal  arrangement  for  research 
participation  by  ROTC  cadets,  law  enforcement  officers,  or  military  personnel. 

Instructional  Strategies 

Before  data  collection  began,  we  assumed  that  a  unique  advantage  of  the  demonstration 
was  that  it  clearly  conveyed  an  appropriate  mission  tempo,  that  is  the  appropriate  speed  to 
perform  the  procedures.  The  performance  measure  of  number  of  rooms  searched,  and  our 
subjective  evaluations  of  team  performance,  indicated  that  on  average,  teams  not  in  the 
demonstration  group  actually  assumed  a  faster  tempo  than  the  demonstration  group.  The  steady 
tempo  conveyed  in  the  demonstration  apparently  emphasized  accuracy  over  speed.  It  is  also 
possible  that  the  demonstration  teams  performed  more  slowly  because  the  had  less  practice  on 
the  task  than  did  the  coaching  and  control  groups.  Nevertheless,  for  situations  in  which  a  faster 
tempo  is  desired,  a  demonstration  could  start  off  at  a  slower  pace,  so  that  the  procedures  are  easy 
to  observe  by  the  trainee,  and  then  the  tempo  is  increased. 

We  also  observed  that  team  members  in  the  demonstration  group  not  only  remembered 
the  mission  procedures  from  the  demonstration,  but  also  closely  modeled  specific  aspects  of  the 
voice  communications.  For  example,  when  announcing  it  was  time  to  exit  the  building,  several 
of  the  Team  Leaders  used  phrasing  (not  specified  in  the  training  manual)  very  similar  to  what 
they  had  heard  in  the  demonstration.  Although  more  than  90%  of  the  demonstration  teams 
attempted  to  exit  the  building,  only  about  58  exited  in  time.  Thus,  the  demonstration  clearly 
presented  a  highly  memorable  example  of  announcing  it  was  time  to  exit,  but  did  not  convey 
with  equivalent  effectiveness  the  higher  level  skills,  such  as  monitoring  time  and  avoiding 
distractions,  needed  to  successfully  exit  within  the  time  limit. 

An  implication  of  the  observation  that  the  participants  imitated  closely  the  tempo  and 
voice  communications  is  that  a  demonstration  should  not  include  any  aspect  of  behavior  that  you 
do  not  want  the  trainees  to  copy. 

The  demonstration  intervention  is  by  far  the  easiest  to  administer.  After  the  initial  effort 
to  produce  and  record  the  demonstration  mission,  it  is  easy  to  administer  and  makes  sure  that 
each  team  receives  the  same  information  before  their  first  practice  mission. 

Coaching  was  the  most  difficult  intervention  to  administer.  In  some  situations  it  was  very 
valuable  to  be  able  to  correct  a  mistake  the  first  time  it  happened  rather  than  watch  a  procedure 
being  practiced  incorrectly  throughout  a  mission  as  could  occur  in  the  other  groups.  However, 
there  was  a  tendency  to  provide  too  much  feedback  too  soon.  In  addition,  because  performance 
at  the  beginning  of  the  first  practice  mission  tended  to  include  many  errors,  coaching  was 
vulnerable  to  being  too  negative. 

The  design  of  our  VEs,  in  contrast  to  open  terrain  or  larger  building  models,  took  away 
one  potential  advantage  of  coaching.  The  teams  could  not  get  lost  or  stray  outside  their  assigned 


28 


sector  because  the  doorway  leading  beyond  their  sector  was  locked.  An  attempt  to  go  through 
that  doorway  was  scored  as  an  error  but  they  could  not  waste  a  significant  amount  of  the  mission 
time  becoming  lost,  realizing  they  are  lost,  and  working  their  way  back  to  where  they  should  be. 
Coaching  would  allow  intervention  at  some  point  during  this  “lost  patrol”  situation,  reducing  the 
amount  of  misspent  training  time. 

In  contrast  to  the  coaching  group  in  which  the  experimenter  had  to  provide  feedback 
immediately,  the  critique  given  during  tihe  replay  benefited  from  the  context  of  having  seen  the 
entire  mission  beforehand.  An  approach  that  seemed  to  work  well  was  to  make  only  one  or  two 
points  per  room.  In  that  way,  by  the  end  of  the  replay  most  problems  could  be  addressed  and 
examples  of  good  performance  could  also  be  noted. 

Here  are  some  examples  of  mistakes  which  were  frequently  pointed  out  during  both 
coaching  missions  and  mission  replays.  The  Team  Leader  would  often  start  the  stack  standing 
too  far  away  from  the  door  about  to  be  entered.  This  does  not  allow  the  Equipment  Specialist 
enough  room  to  stack  behind  the  Team  Leader.  For  either  coaching  or  replay,  the  experimenter 
would  point  out  this  error.  Another  frequent  mistake  involved  initial  contact  with  OPFOR  when 
entering  a  room.  Some  Team  leaders  would  freeze  in  the  doorway,  preventing  the  Equipment 
Specialist  fi-om  entering  the  room.  Other  frequent  mistakes  involved  forgetting  to  use  the  radio 
c^l  signs  and  forgetting  to  switch  to  the  pistol  after  using  the  paint  marker.  “Keep  track  of  the 
elapsed  time”  was  a  frequent  admonition. 

That  at  least  some  of  the  control  group  teams  did  well  indicated  that  the  mission  manual 
contained  adequate  information  and  that  practice  without  any  external  guidance  could  lead  to 
improved  performance.  For  other  control  group  teams,  performance  actually  deteriorated  as  they 
forgot  procedures  or  developed  nonproductive  or  counterproductive  variations  of  the  procedures. 

These  instruction  strategies  are  not  mutually  exclusive.  We  expect  that  an  effective  VE 
training  system  would  use  aspects  of  all  of  these  strategies. 

There  are  a  number  of  factors  which  may  have  contributed  to  the  failure  to  find 
statistically  significant  differences  in  performance  scores  between  the  control  group  and  the  three 
instructional  strategy  conditions.  The  first  was  the  high  variability  in  performance  among  teams 
even  in  the  same  treatment  condition.  The  participant  population  was  highly  heterogeneous, 
including  introductory  psychology  students  and  advanced  ROTC  students,  a  wide  range  of  ages, 
and  teams  composed  of  members  who  were  sometimes  close  friends,  sometimes  complete 
strangers,  and  sometimes  even  parent  and  adult  child.  Whereas  some  teams  were  truly  naive 
teams,  others  included  a  confederate. 

The  second  factor  was  the  multidimensional  nature  of  the  task,  which  allowed  teams  the 
flexibility  to  give  greater  emphasis  to  different  aspects  of  the  task,  such  as  speed  of  search  or 
accuracy. 

The  third  factor  was  the  effectiveness  of  the  control  group  training.  Control  group 
performance  improved  over  the  three  missions,  despite  the  fact  that  those  teams  were  given  no 
feedback  other  than  that  intrinsic  to  the  task.  This  intrinsic  feedback,  combined  with  the  fact  that 
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the  control  group  received  more  task  practice  than  all  but  the  coaching  group  teams,  was 
apparently  enough  to  boost  their  performance  to  the  level  of  the  instructional  strategy  groups. 

The  results  might  have  been  very  different  if  the  groups  had  been  equated  on  mission  time,  rather 
than  total  training  time, 

Finally,  it  should  be  noted  that  the  teams  actually  had  very  little  exposure  to  the 
instmctional  strategies.  The  demonstration  and  replay  sessions  lasted  eight  minutes  each. 
Coaching  was  provided  during  two  eight-minute  missions,  while  the  team  was  actually  engaged 
in  performing  the  mission. 

Presence 


We  had  expected,  but  did  not  find,  that  the  Presence  questionnaire  scores  would  be  higher 
than  previous  ARI VE  experiments.  This  expectation  was  based  upon  the  use  in  this  experiment 
of  VEs  and  tasks  that  are  more  complex,  the  presence  of  a  teammate,  and  the  use  of  sound 
effects.  In  addition,  several  of  the  participants  reported  that  they  felt  highly  involved  in  the 
missions  to  that  extent  that  they  became  nervous.  It  may  be  that  the  high  complexity  of  the 
experimental  task  so  engrossed  the  participants  in  trying  to  recall  the  correct  procedures  that  they 
did  not  attend  to  the  other  aspects  of  the  VE.  For  VE  training  research,  significant  challenges 
remain  as  to  how  to  define  and  measure  the  sense  of  presence. 


Future  Research 

Technical  lessons  learned  from  this  experiment  were  used  in  the  development  of  a 
follow-on  system  that  will  be  used  in  research  to  examine  distributed  team  training  in  which  the 
participants  are  at  remote  sites.  Lessons  learned  concerning  the  instructional  strategies  will  be 
incorporated  in  a  Science  and  Technology  Objective  (STO)  involving  Virtual  Environments  for 
Dismounted  Soldier  Simulation,  Training,  and  Mission  Rehearsal. 
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Appendix  A.  Acronyms 


AAR 

ARI 

CGES 

DIS 

FITT 

HAZMAT 

HMD 

ICS 

UQ 

MCO 

OPFOR 

PC 

PDU 

PQ 

RE2 

ROTC 

SGI 

SSQ 

STO 

STRICOM 

SWAT 

VCR 

VE 

VEPAB 


After  Action  Review 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

Computer  Generated  Entity  Server 

Distributed  Interactive  Simulation 

Fully  Immersive  Team  Training 

Hazardous  Materials 

Head  Mounted  Display 

Individual  Combatant  Simulator 

Immersive  Tendencies  Questionnaire 

Multi  Channel  Option 

Opposing  Forces 

Personal  Computer 

Protocol  Data  Unit 

Presence  Questionnaire 

Reality  Engine  2 

Reserve  Officer  Training  Corps 

Silicon  Graphics,  Inc 

Simulator  Sickness  Questionnaire 

Science  and  Technology  Objective 

Simulation,  Training  and  Instmmentation  Command 

Special  Weapons  and  Tactics  Teams 

Video  Cassette  Recorder 

Virtual  Environment 

Virtual  Environment  Performance  Assessment  Battery 
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Hazardous  Materials  (HAZMAT) 
Team  Training  Manual 


Duty  Position:  Team  Leader 


Team  Leader  Trainee 


(name) 


(date) 
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Team  Task  Overview 


A  two-person  team  searches  rooms  of  a  building  for  canisters  containing  hazardous  gas.  The 
team  carries  equipment  to  determine  if  a  canister  contains  gas  and  to  deactivate  the  canisters.  The 
search  is  complicated  by  the  threat  of  interference  by  looters  and  terrorists.  Each  team  member 
carries  a  tranquilizer  dart  gun  (Pistol)  which  can  temporarily  incapacitate  the  enemy.  Air  tank 
capacity  limits  the  search  time  to  a  maximum  of  8  minutes  per  mission.  Simulated  speed  of 
movement  is  at  most  a  fast  walk. 

Team  member  #1  is  the  designated  Team  Leader.  The  Team  Leader  directs  team  movement  to 
provide  an  efficient  search  while  maintaining  team  security.  The  leader  radios  reports  of 
deactivations  and  encounters  with  the  enemy  to  an  off-site  Mission  Commander.  In  addition  to  a 
Pistol,  the  Team  Leader  carries  a  Paint  Marker  used  to  mark  rooms  that  have  been  searched. 

Team  member  #2  is  the  HAZMAT  (hazardous  materials)  Equipment  Specialist.  In  addition  to  a 
Pistol,  the  Equipment  Specialist  carries  a  device  to  detect  if  a  canister  is  active  and  to  deactivate 
the  canister. 

You  will  have  several  opportunities  to  practice  the  procedures  described  in  this  manual.  The 
other  members  of  your  team  will  count  on  you  to  follow  these  procedures. 

The  overall  team  performance  measure  is  the  number  of  canisters  deactivated  and  reported. 
Failure  to  exit  the  building  before  air  supplies  expire  or  major  breakdowns  in  team  security 
procedures  will  be  scored  as  failed  missions. 


Learning  Objectives 

In  this  manual  you  will  learn: 

The  rule  for  the  order  in  which  to  search  rooms  and  how  to  keep  track  of  which  rooms  you  have 
already  searched. 

How  your  team  should  move  through  hallways  and  enter  rooms. 

When  and  how  to  communicate  using  appropriate  radio  call  signs. 

What  to  do  when  you  encounter  ENEMY. 

A  general  idea  of  the  responsibilities  of  your  other  team  member. 
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HAZMAT  Team  Member 


Suit  and  body  armor 
provide  protection  but 
limit  speed  of  movement 
to  walking  only. 

No  running,  crawling, 
or  jumping! 
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Team  Leader  Duties  Overview 


The  primary  duties  of  the  Team  Leader  involve  search,  communications,  and  security 
procedures. 

Search: 


1)  use  the  right-turn  rule  (explained  later)  for  efficient  search  of  an  assigned  section  of 
the  building 

2)  use  the  Paint  Marker  to  mark  a  room  after  your  team  searches  it 


Paint  Marker 
Communication: 

1)  inform  the  off-site  Commander  of  your  mission  progress 

2)  coordinate  search  and  security  with  your  team  member 
Security: 

1)  move  through  hallways  and  doorways  in  proper  team  formation 

2)  use  your  Tranquilizer  Pistol  to  engage  enemy  quickly  and  decisively 


Tranquilizer  Pistol 
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The  Right-Turn  Search  Rule 


The  right  turn  rule  is  a  rule  of  thumb  to  help  you  organize  your  search  pattern.  At  any  point  in 
the  building  where  you  have  a  choice  of  directions,  turn  right  if  possible.  Look  at  Figure  1.  For 
this  example  assume  that  you  start  your  mission  in  room  #1 .  After  searching  that  room  you 
would  exit  through  the  doorway  on  the  east  wall.  Use  the  paint  marker  to  put  an  “X”  above  the 
doorway,  indicating  that  you  have  searched  that  room.  Exercising  the  right  turn  rule,  you  would 
turn  right  (south)  and  proceed  to  the  end  of  the  hallway.  The  hallway  is  a  dead  end,  so  you  would 
turn  around  and  proceed  north.  You  will  pass  the  room  you  just  searched  and  come  to  a  doorway 
on  the  right  side  of  the  hallway.  Exercising  the  right  turn  rule,  you  would  search  this  room  (#2) 
and  mark  the  doorway  when  you  exit.  Proceeding  north,  there  is  a  doorway  on  the  left  side  of  the 
hallway.  Because  it  is  on  the  left,  you  will  pass  that  room  for  now.  Proceeding  north,  you  reach 
the  end  of  the  hallway.  There  is  a  doorway  straight  ahead  to  the  north  and  the  hallway  turns  to 
the  right  (East).  Using  the  right  turn  rule,  you  would  turn  right  and  proceed  to  the  next  doorway 
(room#3).  Exit  3  and  mark  the  doorway.  A  right  turn  takes  you  to  room  #4.  After  4  the  hallway  is 
a  dead  end.  Turn  around  and  proceed  west.  The  doorway  on  the  left  (Room#4)  will  be  marked 
with  an  X  because  you  already  searched  that  room.  Search  and  mark  #5,  and  #6.  When  you  exit 
room  6  you  could  turn  left  or  go  straight  ahead.  You  have  already  searched  the  rooms  to  the  left. 
Go  straight  (south) .  Search  and  mark  the  room  on  the  right  (#7).  Proceed  south  to  room  #1  .  You 
have  finished  the  search  for  this  building. 

Example  of  Room  Search  Sequence 
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Movement  Procedures 


Hallway  Movement  Technique 

The  Team  will  move  through  hallways  in  single  file  with  the  Team  Leader  in  front.  The 
Equipment  Specialist  should  walk  about  10  feet  behind  the  Team  Leader.  The  Equipment 
Specialist  guards  the  team’s  back  by  frequently  looking  to  the  rear.  The  Team  Leader  should  not 
get  too  far  ahead  of  the  Equipment  Specialist. 

Room  Entry  and  Exit 


The  team  is  most  vulnerable  to  ENEMY  attack  when  entering  and  exiting  rooms. 

ENTRY 

The  Team  Leader  should  move  to  face  the  door  of  the  room  being  careful  not  to  activate  the 
door  before  the  team  is  ready  to  enter.  The  Team  Leader  says  “Stack”  telling  the  Equipment 
Specialist  to  move  directly  behind  the  Team  Leader.  The  Equipment  Specialist  says  “Ready” 
when  in  position.  The  Equipment  Specialist  does  not  need  to  be  exactly  in  line  with  the  Team 
Leader,  but  should  be  aligned  to  move  through  the  doorway.  The  Team  Leader  says  “Go”  and 
moves  forward  to  activate  the  door.  The  Team  Leader  must  move  far  enough  into  the  room  to 
allow  the  Equipment  Specialist  to  move  into  the  room.  The  Team  Leader  should  initially  scan 
the  left  side  of  the  room  and  the  Equipment  Specialist  should  scan  to  the  right. 
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EXIT 


Before  the  team  leaves  the  room  the  Equipment  Specialist  should  select  the  Pistol. 

The  team  should  exit  the  room  in  single  file,  with  the  team  leader  in  front.  In  the  hallway,  the 
Equipment  Specialist  will  watch  for  ENEMY  while  the  Team  Leader  selects  the  Paint  Marker, 
marks  the  room  just  searched,  and  then  selects  the  Pistol.  The  Team  Leader  will  lead  the  team  to 
the  next  room  to  be  searched. 


Communications  Procedures 


Each  HAZMAT  helmet  contains  a  radio  set  with  headphones  and  a  voice  actuated  microphone. 
To  talk  over  the  radio  just  talk  out  loud.  The  radio  set  has  no  switches  or  manual  controls. 

Radio  Call  signs: 

Sierra:  Mission  Commander 

Green  1 :  Team  Leader 

Green2:  Equipment  Specialist 

Except  for  a  communications  check  at  the  beginning  of  each  mission  (explained  later),  team 
members  will  usually  not  use  call  signs  when  talking  to  each  other. 

The  Team  Leader  will  report  mission  progress  to  the  Mission  Commander.  The  Team  Leader 
should  always  use  the  Commander’s  call  sign  “Sierra”  ( see  -  air  -  ah)  at  the  beginning  of  any 
message  for  the  Commander.  The  Equipment  Specialist  never  talks  to  the  Commander. 
Examples  of  radio  communications  are  given  later  in  this  manual. 
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Actions  on  Contact 


During  the  mission  your  team  may  encounter  ENEMY. 


ENEMY  may  be : 


lightly  armed  looters 


heavily  armed  terrorists 


You  may  also  encounter 


An  innocent  bystander  (neutral). 
An  innocent  bystander  is  reported  as  a  “neutral”. 
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Actions  on  Contact  (continued) 


When  you  first  spot  an  ENEMY  you  should  alert  your  partner  by  announcing  “Contact  Front !” 
(or  Left,  Right,  Rear  as  appropriate).  Do  NOT  assume  that  your  partner  has  seen  the  ENEMY. 
Talk  to  each  other!  Shoot  the  ENEMY  using  your  Pistol.  Do  not  point  your  Pistol  at  your  team 
mate  or  risk  a  shot  that  may  hit  your  team  mate  or  a  member  of  another  HAZMAT  team  that  has 
mistakenly  entered  your  area.  Be  sure  of  your  target  before  you  shoot. 

After  each  encounter  with  the  ENEMY  the  Team  Leader  should  report  to  the  mission 
Commander; 

Team  Leader:  “Sierra”  [Commander’s  radio  call  sign] 

“this  is  Greenl”  [your  call  sign] 

wait  for  Commander  to  answer 

Commander:  “Greenl,  this  is  Sierra” 

the  Team  Leader  will  state:  the  number  of  ENEMY  encountered,  the  kind  of  ENEMY,  and  the 
results  of  engagement 

Team  Leader;  “Sierra,  Engaged  and  neutralized  2  lightly  armed  looters,  over” 
Commander;  “Good  work  Green  Team,  continue  mission” 

Shoot  any  ENEMY  in  a  room  before  trying  to  find  or  deactivate  canisters.  The  team  leader  must 
exercise  judgment  in  that  having  each  team  member  search  separate  sections  of  the  room  will 
produce  a  faster  search  but  sacrifices  security. 

The  Team  Leader  should  report  to  the  Conunander  when  your  team  sees  an  innocent  bystander 
(a  neutral).  You  do  not  have  equipment  or  time  to  help  bystanders.  A  medical  unit  will  be  sent  to 
evacuate  them  after  you  have  finished  your  mission. 
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Deactivating  Canisters 


When  a  canister  is  found  the  Equipment  Specialist  will  use  the  detector/deactivator  (DELTA2)  to 
determine  if  the  canister  is  active  and  if  so,  to  deactivate  it.  When  the  Delta2  is  within  about  5 
feet  of  a  canister  the  display  will  turn  yellow  if  the  canister  is  active.  The  display  will  turn  red 
when  within  deactivation  range.  When  the  display  is  red  the  Equipment  Specialist  presses  the 
trigger  to  deactivate  the  canister  and  the  display  will  then  turn  green  indicating  a  successful 
deactivation.  If  the  Delta2  display  does  not  register  yellow  or  red  near  a  canister  that  canister  is  a 
“dud”.  That  is,  it  does  not  contain  hazardous  gas.  The  Equipment  Specialist  should  inform  the 
Team  Leader  of  the  status  of  each  canister. 


Beyond  about  5  feet,  the  Delta2  display 
is  “green”  because  gas,  if  any,  is  out  of 
range. 

At  about  2  feet,  the  display  turns 
Red  if  the  canister  is  active. 

When  the  Equipment  Specialist 
triggers  the  Delta2,  the  display  turns 
green,  indicating  deactivation. 


Before  leaving  a  room  the  Team  Leader  will  report  canisters  that  were  detected. 


(Team  Leader) 
(Commander) 
(Team  Leader) 
(Commander) 


“Sierra,  this  is  Green  1” 

“Green  1,  go  ahead” 

“Found  two  canisters,  deactivated  one,  the  other  was  a  dud.” 
“Continue  mission” 


Mission  Preparation 


You  begin  each  mission  from  the  back  of  an  armored  van.  The  letters  PRIMED  are  on  the  inside 
of  the  van  door  to  give  you  something  to  focus  on  while  you  make  final  adjustments  to  your 
helmet. 

Both  team  members  should  check  that  their  countdown  timers  are  at  “8:00”,  select  the  Pistol,  and 
face  the  van  door.  The  Team  Leader  will  conduct  a  communications  (commo)  check  with  the 
Equipment  Specialist: 

(Team  Leader):  “Green2,  this  is  Green  1,  commo  check” 

(Equipment  Specialist):  “Green  1,  loud  and  clear” 

The  Team  Leader  will  tell  the  mission  Commander  that  the  team  is  ready: 

(Team  Leader):  “Sierra,  Green  Team  is  ready” 

(Commander):  “Green  Team,  begin  mission” 

The  van  has  been  backed  up  to  an  opening  in  the  building  you  are  to  search.  The  van  door  will 
open  onto  the  first  room  to  be  searched.  After  both  team  members  exit  the  van  the  door  will 
close.  Before  eight  minutes  expire  the  team  should  be  back  at  the  van  door.  As  the  team 
approaches  the  door  the  Team  Leader  should  inform  the  Commander  that  the  team  is  at  the  door 
(the  Commander  will  open  the  door  by  remote  control).  The  Team  Leader  should  tell  the 
Commander  when  both  team  members  are  back  inside  the  van. 


Outside  view  of  van  door 


Miscellaneous 


The  environment  in  which  you  will  work  is  so  noisy  you  must  wear  (simulated)  hearing 
protection  that  blocks  ALL  outside  noise.  Therefore,  you  can  talk  to  your  teammate  and/or 
mission  controller  only  by  the  radio  headset.  You  can  not  hear  ENEMY,  even  if  they  shoot  at 
you.  If  you  are  hit  by  ENEMY  bullets,  collide  with  objects,  or  open  doors  you  may  hear  sound 
conducted  within  your  suit. 

Although  your  HAZMAT  suit  and  body  armor  protect  you  from  ENEMY  weapons,  you  must 
deal  quickly  with  ENEMY  or  else  they  may  damage  your  equipment  causing  the  mission  to  fail. 

Do  not  attempt  to  move  anything  such  as  furniture  or  debris.  Because  of  the  heat  and  corrosive 
gas  emitted  by  an  active  canister,  a  canister  can  not  be  hidden  in  or  under  anything. 

Your  Tranquilizer  Pistol  contains  10  darts  per  mission. 

The  Paint  Marker  contains  enough  paint  for  about  12  charges  per  mission. 

The  Delta2  has  enough  battery  power  to  deactivate  about  10  canisters  per  mission.  Attempting  to 
deactivate  a  canister  that  is  already  inactive  wastes  the  battery  of  the  Delta2. 

If  both  team  members  shoot  at  and  hit  the  same  individual  ENEMY  that  is  at  best  a  waste  of 
ammunition  and  may  complicate  the  work  of  the  medical  recovery  team.  However,  it  would  be 
better  for  both  to  shoot  than  to  have  neither  shoot  and  allow  the  ENEMY  to  shoot  you. 

Shooting  ENEMY  with  your  paint  marker  is  a  waste  of  paint  and  is  scored  as  an  error. 

Do  not  cross  into  the  search  area  of  other  teams.  If  other  teams  mistakenly  enter  your  area, 
contact  your  Commander.  The  other  teams  use  different  radio  frequencies,  therefore  you  can  not 
communicate  directly  with  them. 


Review: 

Sierra: 

Mission  Commander 

Green  1: 

Team  Leader 

Green2: 

Equipment  Specialist 

Delta  2  : 

detector/deactivator  device 

pistol: 

tranquilizer  dart  gun 

marker: 

paint  spray  device 

canister: 

hazardous  materials  container 

a  canister  may  be  active:  (live) 
a  canister  may  be  inactive:  no  longer  a  threat  (dud) 
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P  Preparation:  conduct  radio  and  air  supply  timer  checks 

R  Right-turn  rule:  search  rooms  in  correct  order 

I  Inform  your  team  mate  and  Commander  about  ENEMY 
and  canisters 

M  Movement  technique:  move  single  file  and  “stack”  before 
room  entry 

E  Exit  building  (return  to  van)  before  time  expires 
D  Detect  and  deactivate  canisters  (primary  goal  of  mission!) 
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